[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:38:57 dist-test-slave-1x4g krb5kdc[20273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:38:57 dist-test-slave-1x4g krb5kdc[20273](info): set up 2 sockets
Aug 10 22:38:57 dist-test-slave-1x4g krb5kdc[20273](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 22:38:58.115255 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:38:58 dist-test-slave-1x4g krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:38:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:38:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:38:58.236021 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34843
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:38:58.380431 20287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:38:58.380714 20287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:38:58.380846 20287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:38:58.384887 20287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:38:58.384968 20287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:38:58.385022 20287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:38:58.385067 20287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:38:58.385102 20287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:38:58.390156 20287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:38:58.391546 20287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:38:58.392844 20287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:38:58.405882 20298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:38:58.406280 20295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:38:58.406288 20294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:38:58.406579 20287 server_base.cc:1034] running on GCE node
I20260810 22:38:58.407264 20287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:38:58.408869 20287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:38:58.410060 20287 hybrid_clock.cc:648] HybridClock initialized: now 1786401538410034 us; error 36 us; skew 500 ppm
Aug 10 22:38:58 dist-test-slave-1x4g krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401538, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:38:58.413009 20287 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:38:58.414079 20287 webserver.cc:460] Webserver started at http://127.19.202.126:35461/ using document root <none> and password file <none>
I20260810 22:38:58.414927 20287 fs_manager.cc:359] Metadata directory not provided
I20260810 22:38:58.415020 20287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:38:58.415406 20287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:38:58.418202 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "7c21902a2ed044d7a1d95c0e1da4547a"
format_stamp: "Formatted at 2026-08-10 22:38:58 on dist-test-slave-1x4g"
server_key: "c0a7474b6d4d17eeb997de3ff35364fd"
server_key_iv: "03de6391223064632f9579d3dbfe3b48"
server_key_version: "encryptionkey@0"
I20260810 22:38:58.418773 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "7c21902a2ed044d7a1d95c0e1da4547a"
format_stamp: "Formatted at 2026-08-10 22:38:58 on dist-test-slave-1x4g"
server_key: "c0a7474b6d4d17eeb997de3ff35364fd"
server_key_iv: "03de6391223064632f9579d3dbfe3b48"
server_key_version: "encryptionkey@0"
I20260810 22:38:58.423733 20287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 22:38:58.427241 20305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:58.428791 20287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:38:58.429065 20287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "7c21902a2ed044d7a1d95c0e1da4547a"
format_stamp: "Formatted at 2026-08-10 22:38:58 on dist-test-slave-1x4g"
server_key: "c0a7474b6d4d17eeb997de3ff35364fd"
server_key_iv: "03de6391223064632f9579d3dbfe3b48"
server_key_version: "encryptionkey@0"
I20260810 22:38:58.429185 20287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:38:58.447683 20287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:38:58.451685 20287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:38:58.451951 20287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:38:58.469161 20287 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:34843
I20260810 22:38:58.469393 20365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:34843 every 8 connection(s)
I20260810 22:38:58.470526 20287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:38:58.472832 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 20287
I20260810 22:38:58.472998 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:38:58.474491 20265 external_mini_cluster.cc:1428] Setting key ea8d6d6147673dc493bdf415d9794ed7
I20260810 22:38:58.475536 20366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:38:58.485409 20366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a: Bootstrap starting.
I20260810 22:38:58.489631 20366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a: Neither blocks nor log segments found. Creating new log.
I20260810 22:38:58.491215 20366 log.cc:824] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a: Log is configured to *not* fsync() on all Append() calls
I20260810 22:38:58.494400 20366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a: No bootstrap required, opened a new log
Aug 10 22:38:58 dist-test-slave-1x4g krb5kdc[20273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:38:58.501847 20366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34843 } }
I20260810 22:38:58.502254 20366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:38:58.502339 20366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c21902a2ed044d7a1d95c0e1da4547a, State: Initialized, Role: FOLLOWER
I20260810 22:38:58.503135 20366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34843 } }
I20260810 22:38:58.503371 20366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:38:58.503509 20366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:38:58.503645 20366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:38:58.505043 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:58.479328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.19.202.126:34843)
0810 22:38:58.479827 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0810 22:38:58.479864 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0810 22:38:58.481002 (+ 1138us) server_negotiation.cc:407] Connection header received
0810 22:38:58.485665 (+ 4663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:38:58.485682 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:38:58.486042 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:38:58.486354 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:38:58.488824 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:58.490134 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:58.491332 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:58.491959 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:58.492147 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:58.497967 (+ 5820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:38:58.498011 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:38:58.498028 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:38:58.498074 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:38:58.500878 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:58.501405 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:58.501411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:58.501417 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:58.501610 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:58.501928 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:58.501931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:58.501933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:58.502617 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:38:58.502921 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 22:38:58.503417 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 22:38:58.503758 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":166,"threads_started":1}
I20260810 22:38:58.505137 20366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34843 } }
I20260810 22:38:58.505792 20366 leader_election.cc:302] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c21902a2ed044d7a1d95c0e1da4547a; no voters:
I20260810 22:38:58.506250 20366 leader_election.cc:288] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:38:58.506441 20371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:38:58.506894 20371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [term 1 LEADER]: Becoming Leader. State: Replica: 7c21902a2ed044d7a1d95c0e1da4547a, State: Running, Role: LEADER
I20260810 22:38:58.507403 20371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34843 } }
I20260810 22:38:58.507769 20366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:38:58.509431 20374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c21902a2ed044d7a1d95c0e1da4547a. Latest consensus state: current_term: 1 leader_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34843 } } }
I20260810 22:38:58.509610 20374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [sys.catalog]: This master's current role is: LEADER
I20260810 22:38:58.510641 20381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:38:58.511145 20373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c21902a2ed044d7a1d95c0e1da4547a" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34843 } } }
I20260810 22:38:58.511250 20373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a [sys.catalog]: This master's current role is: LEADER
I20260810 22:38:58.516923 20381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:38:58.525420 20381 catalog_manager.cc:1347] Generated new cluster ID: d54d9d066a8b438186a22a1e462df351
I20260810 22:38:58.525490 20381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:38:58.540506 20381 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:38:58.541662 20381 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:38:58.547320 20381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c21902a2ed044d7a1d95c0e1da4547a: Generated new TSK 0
I20260810 22:38:58.548086 20381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:38:58.592422 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34843
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:38:58.766273 20395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:38:58.766507 20395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:38:58.766582 20395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:38:58.770504 20395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:38:58.770557 20395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:38:58.770660 20395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:38:58.775962 20395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:38:58.777097 20395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:38:58.778091 20395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:38:58.789460 20401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:38:58.789508 20402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:38:58.789671 20395 server_base.cc:1034] running on GCE node
W20260810 22:38:58.789847 20404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:38:58.790256 20395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:38:58.790800 20395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:38:58.791987 20395 hybrid_clock.cc:648] HybridClock initialized: now 1786401538791973 us; error 29 us; skew 500 ppm
Aug 10 22:38:58 dist-test-slave-1x4g krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401538, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:38:58.795322 20395 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:38:58.796109 20395 webserver.cc:460] Webserver started at http://127.19.202.65:42101/ using document root <none> and password file <none>
I20260810 22:38:58.796650 20395 fs_manager.cc:359] Metadata directory not provided
I20260810 22:38:58.796710 20395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:38:58.796931 20395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:38:58.799084 20395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "adcbc717ae7143a0a975e98cb2650b0a"
format_stamp: "Formatted at 2026-08-10 22:38:58 on dist-test-slave-1x4g"
server_key: "b3c331a2b1a0e719f979e6a9688e1dde"
server_key_iv: "fbb60cf9adb6aa4ad1d949bf2a16d034"
server_key_version: "encryptionkey@0"
I20260810 22:38:58.799697 20395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "adcbc717ae7143a0a975e98cb2650b0a"
format_stamp: "Formatted at 2026-08-10 22:38:58 on dist-test-slave-1x4g"
server_key: "b3c331a2b1a0e719f979e6a9688e1dde"
server_key_iv: "fbb60cf9adb6aa4ad1d949bf2a16d034"
server_key_version: "encryptionkey@0"
I20260810 22:38:58.803687 20395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 22:38:58.806123 20411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:58.807299 20395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:38:58.807410 20395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "adcbc717ae7143a0a975e98cb2650b0a"
format_stamp: "Formatted at 2026-08-10 22:38:58 on dist-test-slave-1x4g"
server_key: "b3c331a2b1a0e719f979e6a9688e1dde"
server_key_iv: "fbb60cf9adb6aa4ad1d949bf2a16d034"
server_key_version: "encryptionkey@0"
I20260810 22:38:58.807505 20395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:38:58.833758 20395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:38:58.837343 20395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:38:58.837605 20395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:38:58.838461 20395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:38:58.839689 20395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:38:58.839757 20395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:58.839826 20395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:38:58.839871 20395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:58.858826 20395 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:35915
I20260810 22:38:58.858976 20536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:35915 every 8 connection(s)
I20260810 22:38:58.859764 20395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:38:58.860582 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 20395
I20260810 22:38:58.860690 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:38:58.861008 20265 external_mini_cluster.cc:1428] Setting key 99e91b889b8acd33d353cc8342a437f4
Aug 10 22:38:58 dist-test-slave-1x4g krb5kdc[20273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401538, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:38:58.876412 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:58.862159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:59783 (local address 127.19.202.126:34843)
0810 22:38:58.862308 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 22:38:58.862313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:38:58.863802 (+ 1489us) server_negotiation.cc:407] Connection header received
0810 22:38:58.865617 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:38:58.865622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:38:58.865683 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:38:58.865776 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:38:58.867083 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:58.867961 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:58.869311 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:58.869741 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:58.869856 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:58.873127 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:38:58.873130 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:38:58.873131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:38:58.873153 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:38:58.874500 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:58.874926 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:58.874928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:58.874929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:58.875005 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:58.875317 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:58.875319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:58.875320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:58.875495 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:38:58.875583 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 22:38:58.876150 (+ 567us) server_negotiation.cc:299] Negotiation successful
0810 22:38:58.876285 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:38:58.877650 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:58.862623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34843 (local address 127.19.202.65:59783)
0810 22:38:58.863636 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0810 22:38:58.863696 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:38:58.865386 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:38:58.865947 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:38:58.865954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:38:58.866369 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:38:58.866929 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:38:58.866943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:38:58.868105 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:38:58.868108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:38:58.869190 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:38:58.869199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:38:58.869888 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:38:58.869891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:38:58.870069 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:58.870793 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:38:58.870809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:38:58.872928 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:38:58.874603 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:38:58.874607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:38:58.874617 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:38:58.874844 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:38:58.875098 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:38:58.875100 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:38:58.875102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:38:58.875233 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:38:58.875611 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:38:58.875616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:38:58.875811 (+ 195us) client_negotiation.cc:769] Sending connection context
0810 22:38:58.876030 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 22:38:58.876364 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":150,"threads_started":1}
I20260810 22:38:58.878919 20537 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34843
I20260810 22:38:58.879137 20537 heartbeater.cc:464] Registering TS with master...
I20260810 22:38:58.879582 20537 heartbeater.cc:511] Master 127.19.202.126:34843 requested a full tablet report, sending...
I20260810 22:38:58.881834 20324 ts_manager.cc:194] Registered new tserver with Master: adcbc717ae7143a0a975e98cb2650b0a (127.19.202.65:35915)
I20260810 22:38:58.884212 20324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:59783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:38:58.893996 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34843
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:38:59.045466 20545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:38:59.045709 20545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:38:59.045780 20545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:38:59.050055 20545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:38:59.050109 20545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:38:59.050215 20545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:38:59.055140 20545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:38:59.056077 20545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:38:59.057111 20545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:38:59.066923 20551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:38:59.067003 20550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:38:59.066999 20553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:38:59.067162 20545 server_base.cc:1034] running on GCE node
I20260810 22:38:59.067672 20545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:38:59.068257 20545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:38:59.069410 20545 hybrid_clock.cc:648] HybridClock initialized: now 1786401539069385 us; error 43 us; skew 500 ppm
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:38:59.072570 20545 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:38:59.073500 20545 webserver.cc:460] Webserver started at http://127.19.202.66:45317/ using document root <none> and password file <none>
I20260810 22:38:59.074033 20545 fs_manager.cc:359] Metadata directory not provided
I20260810 22:38:59.074120 20545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:38:59.074362 20545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:38:59.076367 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "9f35f59b0cd841a9b558fe237d2e7baa"
format_stamp: "Formatted at 2026-08-10 22:38:59 on dist-test-slave-1x4g"
server_key: "94c195ff6195a1e4546d17fbd0653449"
server_key_iv: "2e9a948ef266bb04173228b416e9934e"
server_key_version: "encryptionkey@0"
I20260810 22:38:59.077060 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "9f35f59b0cd841a9b558fe237d2e7baa"
format_stamp: "Formatted at 2026-08-10 22:38:59 on dist-test-slave-1x4g"
server_key: "94c195ff6195a1e4546d17fbd0653449"
server_key_iv: "2e9a948ef266bb04173228b416e9934e"
server_key_version: "encryptionkey@0"
I20260810 22:38:59.080912 20545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 22:38:59.083334 20561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:59.084471 20545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:38:59.084606 20545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "9f35f59b0cd841a9b558fe237d2e7baa"
format_stamp: "Formatted at 2026-08-10 22:38:59 on dist-test-slave-1x4g"
server_key: "94c195ff6195a1e4546d17fbd0653449"
server_key_iv: "2e9a948ef266bb04173228b416e9934e"
server_key_version: "encryptionkey@0"
I20260810 22:38:59.084729 20545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:38:59.097169 20545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:38:59.099938 20545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:38:59.100207 20545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:38:59.100955 20545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:38:59.101884 20545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:38:59.101956 20545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:59.102027 20545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:38:59.102077 20545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:59.120350 20545 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:42021
I20260810 22:38:59.120517 20690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:42021 every 8 connection(s)
I20260810 22:38:59.121249 20545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:38:59.130256 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 20545
I20260810 22:38:59.130338 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:38:59.130600 20265 external_mini_cluster.cc:1428] Setting key beebbfd54bbf8bce7e473dd1fa4f1e63
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:38:59.135850 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:59.123092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:56131 (local address 127.19.202.126:34843)
0810 22:38:59.123231 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:38:59.123234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:38:59.124322 (+ 1088us) server_negotiation.cc:407] Connection header received
0810 22:38:59.125420 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:38:59.125423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:38:59.125481 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:38:59.125565 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:38:59.126900 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.127444 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.128694 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.128988 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.129098 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:59.131861 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:38:59.131867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:38:59.131872 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:38:59.131900 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:38:59.133573 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.134314 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.134327 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.134330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.134467 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.134805 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.134808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.134810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.134992 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:38:59.135094 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 22:38:59.135570 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 22:38:59.135690 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:38:59.136660 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:59.123425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34843 (local address 127.19.202.66:56131)
0810 22:38:59.124190 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 22:38:59.124228 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 22:38:59.125209 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:38:59.125723 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:38:59.125729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:38:59.126176 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:38:59.126744 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:38:59.126759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.127567 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.127571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:38:59.128567 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:38:59.128576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.129133 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.129137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:38:59.129351 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:59.129875 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:38:59.129890 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:38:59.131680 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:38:59.133700 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:38:59.133708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:38:59.133724 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:38:59.134115 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:38:59.134576 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:38:59.134578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:38:59.134580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:38:59.134711 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:38:59.135092 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:38:59.135098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:38:59.135336 (+ 238us) client_negotiation.cc:769] Sending connection context
0810 22:38:59.135523 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 22:38:59.135813 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":164,"threads_started":1}
I20260810 22:38:59.137851 20691 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34843
I20260810 22:38:59.138087 20691 heartbeater.cc:464] Registering TS with master...
I20260810 22:38:59.138540 20691 heartbeater.cc:511] Master 127.19.202.126:34843 requested a full tablet report, sending...
I20260810 22:38:59.139586 20324 ts_manager.cc:194] Registered new tserver with Master: 9f35f59b0cd841a9b558fe237d2e7baa (127.19.202.66:42021)
I20260810 22:38:59.140183 20324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:56131
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:38:59.163260 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34843
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:38:59.314328 20697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:38:59.314579 20697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:38:59.314622 20697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:38:59.318542 20697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:38:59.318598 20697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:38:59.318655 20697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:38:59.324015 20697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:38:59.324954 20697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:38:59.325910 20697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:38:59.337395 20702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:38:59.337411 20706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:38:59.337460 20708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:38:59.337739 20697 server_base.cc:1034] running on GCE node
I20260810 22:38:59.338201 20697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:38:59.338825 20697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:38:59.340014 20697 hybrid_clock.cc:648] HybridClock initialized: now 1786401539339983 us; error 42 us; skew 500 ppm
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:38:59.342782 20697 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:38:59.343618 20697 webserver.cc:460] Webserver started at http://127.19.202.67:39719/ using document root <none> and password file <none>
I20260810 22:38:59.344156 20697 fs_manager.cc:359] Metadata directory not provided
I20260810 22:38:59.344239 20697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:38:59.344458 20697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:38:59.346552 20697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "1041a0f91a274135803f2434f7eac760"
format_stamp: "Formatted at 2026-08-10 22:38:59 on dist-test-slave-1x4g"
server_key: "564afe4de4d41c2f74f2a7ffb7a028fb"
server_key_iv: "e5b7fc83cbd8476a41a54e06c411a1f1"
server_key_version: "encryptionkey@0"
I20260810 22:38:59.347110 20697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "1041a0f91a274135803f2434f7eac760"
format_stamp: "Formatted at 2026-08-10 22:38:59 on dist-test-slave-1x4g"
server_key: "564afe4de4d41c2f74f2a7ffb7a028fb"
server_key_iv: "e5b7fc83cbd8476a41a54e06c411a1f1"
server_key_version: "encryptionkey@0"
I20260810 22:38:59.350924 20697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 22:38:59.353334 20715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:59.354498 20697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:38:59.354632 20697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "1041a0f91a274135803f2434f7eac760"
format_stamp: "Formatted at 2026-08-10 22:38:59 on dist-test-slave-1x4g"
server_key: "564afe4de4d41c2f74f2a7ffb7a028fb"
server_key_iv: "e5b7fc83cbd8476a41a54e06c411a1f1"
server_key_version: "encryptionkey@0"
I20260810 22:38:59.354738 20697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:38:59.376910 20697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:38:59.379660 20697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:38:59.379906 20697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:38:59.380641 20697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:38:59.381522 20697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:38:59.381590 20697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:59.381660 20697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:38:59.381700 20697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:38:59.400817 20697 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:37845
I20260810 22:38:59.400935 20844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:37845 every 8 connection(s)
I20260810 22:38:59.401773 20697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:38:59.409708 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 20697
I20260810 22:38:59.409838 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:38:59.410123 20265 external_mini_cluster.cc:1428] Setting key 7c60d467cefe36055ed88dd59d8a02d1
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:38:59.416326 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:59.403781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:60769 (local address 127.19.202.126:34843)
0810 22:38:59.403904 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 22:38:59.403908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:38:59.405133 (+ 1225us) server_negotiation.cc:407] Connection header received
0810 22:38:59.406259 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:38:59.406262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:38:59.406322 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:38:59.406420 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:38:59.407817 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.408335 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.409809 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.410068 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.410181 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:59.413476 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:38:59.413479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:38:59.413481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:38:59.413504 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:38:59.414636 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.415075 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.415077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.415079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.415149 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.415465 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.415467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.415469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.415630 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:38:59.415704 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:38:59.416129 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 22:38:59.416217 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:38:59.417224 20848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:59.404107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34843 (local address 127.19.202.67:60769)
0810 22:38:59.404988 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 22:38:59.405031 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:38:59.406050 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:38:59.406590 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:38:59.406596 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:38:59.407061 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:38:59.407654 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:38:59.407692 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.408469 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.408472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:38:59.409680 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:38:59.409692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.410218 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.410222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:38:59.410482 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:59.411114 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:38:59.411136 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:38:59.413336 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:38:59.414742 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:38:59.414746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:38:59.414755 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:38:59.414990 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:38:59.415241 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:38:59.415243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:38:59.415244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:38:59.415376 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:38:59.415710 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:38:59.415715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:38:59.415936 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 22:38:59.416071 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 22:38:59.416292 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":171,"threads_started":1}
I20260810 22:38:59.418503 20845 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34843
I20260810 22:38:59.418725 20845 heartbeater.cc:464] Registering TS with master...
I20260810 22:38:59.419185 20845 heartbeater.cc:511] Master 127.19.202.126:34843 requested a full tablet report, sending...
I20260810 22:38:59.420107 20324 ts_manager.cc:194] Registered new tserver with Master: 1041a0f91a274135803f2434f7eac760 (127.19.202.67:37845)
I20260810 22:38:59.420639 20324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:60769
I20260810 22:38:59.424681 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:38:59.453758 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:59.443705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.19.202.65:35915)
0810 22:38:59.444048 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0810 22:38:59.444055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:38:59.444080 (+ 25us) server_negotiation.cc:407] Connection header received
0810 22:38:59.444210 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:38:59.444217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:38:59.444396 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:38:59.444535 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:38:59.445326 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.446199 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.446960 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.447358 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.447471 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:59.450136 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:38:59.450147 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:38:59.450158 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:38:59.450182 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:38:59.451854 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.452307 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.452313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.452314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.452395 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.452663 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.452667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.452669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.452995 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:38:59.453243 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 22:38:59.453423 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 22:38:59.453576 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
W20260810 22:38:59.454761 20442 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42910
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:38:59.484095 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:38:59.475461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.19.202.65:35915)
0810 22:38:59.475580 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 22:38:59.475584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:38:59.475698 (+ 114us) server_negotiation.cc:407] Connection header received
0810 22:38:59.475792 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:38:59.475795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:38:59.475840 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:38:59.475940 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:38:59.476591 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.477400 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.478278 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:38:59.478592 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:38:59.478705 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:38:59.481154 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:38:59.481158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:38:59.481160 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:38:59.481182 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:38:59.482672 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.483066 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.483072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.483077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.483160 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:38:59.483395 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:38:59.483397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:38:59.483399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:38:59.483541 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:38:59.483639 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 22:38:59.483908 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 22:38:59.483991 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:38:59.485177 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 20395
I20260810 22:38:59.492880 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 20545
I20260810 22:38:59.500461 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 20697
I20260810 22:38:59.507551 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 20287
2026-08-10T22:38:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2499 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20861](info): set up 2 sockets
Aug 10 22:38:59 dist-test-slave-1x4g krb5kdc[20861](info): commencing operation
krb5kdc: starting...
W20260810 22:39:00.589615 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:00 dist-test-slave-1x4g krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:00.713901 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41349
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:41349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:00.824872 20875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:00.825117 20875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:00.825204 20875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:00.831357 20875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:00.831429 20875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:00.831462 20875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:00.831501 20875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:00.831537 20875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:00.838531 20875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:41349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:00.839644 20875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:00.840641 20875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:00.849026 20881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:00.849081 20885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:00.849026 20882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:00.849298 20875 server_base.cc:1034] running on GCE node
I20260810 22:39:00.849903 20875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:00.850931 20875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:00.852089 20875 hybrid_clock.cc:648] HybridClock initialized: now 1786401540852078 us; error 25 us; skew 500 ppm
Aug 10 22:39:00 dist-test-slave-1x4g krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401540, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:00.854930 20875 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:00.855760 20875 webserver.cc:460] Webserver started at http://127.19.202.126:35831/ using document root <none> and password file <none>
I20260810 22:39:00.856307 20875 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:00.856401 20875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:00.856654 20875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:00.858706 20875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "96aaccca2f5d426bad9f69b7abf7663f"
format_stamp: "Formatted at 2026-08-10 22:39:00 on dist-test-slave-1x4g"
server_key: "0e03464cab09599efc74abe232cb744b"
server_key_iv: "0a3bf44659b06fcedc66d3576aa21d36"
server_key_version: "encryptionkey@0"
I20260810 22:39:00.859261 20875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "96aaccca2f5d426bad9f69b7abf7663f"
format_stamp: "Formatted at 2026-08-10 22:39:00 on dist-test-slave-1x4g"
server_key: "0e03464cab09599efc74abe232cb744b"
server_key_iv: "0a3bf44659b06fcedc66d3576aa21d36"
server_key_version: "encryptionkey@0"
I20260810 22:39:00.863081 20875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 22:39:00.865338 20892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:00.866402 20875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:00.866549 20875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "96aaccca2f5d426bad9f69b7abf7663f"
format_stamp: "Formatted at 2026-08-10 22:39:00 on dist-test-slave-1x4g"
server_key: "0e03464cab09599efc74abe232cb744b"
server_key_iv: "0a3bf44659b06fcedc66d3576aa21d36"
server_key_version: "encryptionkey@0"
I20260810 22:39:00.866650 20875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:00.898023 20875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:00.901340 20875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:00.901551 20875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:00.908469 20875 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:41349
I20260810 22:39:00.908545 20953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:41349 every 8 connection(s)
I20260810 22:39:00.909463 20875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:00.910673 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 20875
I20260810 22:39:00.910790 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:00.911044 20265 external_mini_cluster.cc:1428] Setting key 24296c66812373b4d65e81c818e15e61
I20260810 22:39:00.912596 20954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:00.919206 20954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f: Bootstrap starting.
Aug 10 22:39:00 dist-test-slave-1x4g krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:00.921779 20954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:00.922730 20954 log.cc:824] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:00.924957 20954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f: No bootstrap required, opened a new log
I20260810 22:39:00.926182 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:00.912472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.19.202.126:41349)
0810 22:39:00.912964 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:00.912982 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:00.913039 (+ 57us) server_negotiation.cc:407] Connection header received
0810 22:39:00.913836 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:00.913858 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:00.914254 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:00.914591 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:00.915197 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:00.916700 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:00.917568 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:00.918270 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:00.918456 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:00.921095 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:00.921101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:00.921114 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:00.921142 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:00.923300 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:00.923745 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:00.923750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:00.923755 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:00.923884 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:00.924126 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:00.924129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:00.924130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:00.924598 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:00.924797 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:00.925036 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 22:39:00.925320 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":139,"threads_started":1}
I20260810 22:39:00.928887 20954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aaccca2f5d426bad9f69b7abf7663f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41349 } }
I20260810 22:39:00.929092 20954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:00.929171 20954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96aaccca2f5d426bad9f69b7abf7663f, State: Initialized, Role: FOLLOWER
I20260810 22:39:00.929559 20954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aaccca2f5d426bad9f69b7abf7663f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41349 } }
I20260810 22:39:00.929701 20954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:00.929786 20954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:00.929919 20954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:00.930794 20954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aaccca2f5d426bad9f69b7abf7663f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41349 } }
I20260810 22:39:00.931144 20954 leader_election.cc:302] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96aaccca2f5d426bad9f69b7abf7663f; no voters:
I20260810 22:39:00.931417 20954 leader_election.cc:288] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:00.931538 20959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:00.931800 20959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [term 1 LEADER]: Becoming Leader. State: Replica: 96aaccca2f5d426bad9f69b7abf7663f, State: Running, Role: LEADER
I20260810 22:39:00.932160 20959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aaccca2f5d426bad9f69b7abf7663f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41349 } }
I20260810 22:39:00.932611 20954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:00.933256 20959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96aaccca2f5d426bad9f69b7abf7663f. Latest consensus state: current_term: 1 leader_uuid: "96aaccca2f5d426bad9f69b7abf7663f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aaccca2f5d426bad9f69b7abf7663f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41349 } } }
I20260810 22:39:00.933368 20959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:00.933784 20967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:00.933902 20960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96aaccca2f5d426bad9f69b7abf7663f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aaccca2f5d426bad9f69b7abf7663f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41349 } } }
I20260810 22:39:00.934068 20960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:00.936852 20967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:00.942327 20967 catalog_manager.cc:1347] Generated new cluster ID: 0d66cdd8ba0d422cbd4468095300884f
I20260810 22:39:00.942410 20967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:00.951932 20967 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:00.952760 20967 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:00.961190 20967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96aaccca2f5d426bad9f69b7abf7663f: Generated new TSK 0
I20260810 22:39:00.961802 20967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:00.995689 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41349
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:01.108302 20984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:01.108562 20984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:01.108633 20984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:01.112664 20984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:01.112715 20984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:01.112835 20984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:01.117882 20984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41349
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:01.118868 20984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:01.119913 20984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:01.129038 20991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:01.129050 20992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:01.129038 20996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:01.129247 20984 server_base.cc:1034] running on GCE node
I20260810 22:39:01.129707 20984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:01.130285 20984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:01.131440 20984 hybrid_clock.cc:648] HybridClock initialized: now 1786401541131422 us; error 29 us; skew 500 ppm
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:01.134495 20984 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:01.135262 20984 webserver.cc:460] Webserver started at http://127.19.202.65:37085/ using document root <none> and password file <none>
I20260810 22:39:01.135762 20984 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:01.135821 20984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:01.135993 20984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:01.138048 20984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "c82543b337024922889b65506ba33d94"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "9ac334a1964729d5d4072ad51239b1fe"
server_key_iv: "bd090b219a5a683aff6bfc330db2df2a"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.138535 20984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "c82543b337024922889b65506ba33d94"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "9ac334a1964729d5d4072ad51239b1fe"
server_key_iv: "bd090b219a5a683aff6bfc330db2df2a"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.142032 20984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:01.144294 21003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.145529 20984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:01.145620 20984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "c82543b337024922889b65506ba33d94"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "9ac334a1964729d5d4072ad51239b1fe"
server_key_iv: "bd090b219a5a683aff6bfc330db2df2a"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.145697 20984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:01.165323 20984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:01.168012 20984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:01.168175 20984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:01.168684 20984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:01.169503 20984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:01.169569 20984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.169612 20984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:01.169636 20984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.179003 20984 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:44815
I20260810 22:39:01.179040 21135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:44815 every 8 connection(s)
I20260810 22:39:01.179874 20984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:01.182183 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 20984
I20260810 22:39:01.182297 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:01.182608 20265 external_mini_cluster.cc:1428] Setting key b0e91e8bbc6d03fffe2d00ff38139bd4
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:39:01.195178 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.181676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:36341 (local address 127.19.202.126:41349)
0810 22:39:01.181840 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.181844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.182874 (+ 1030us) server_negotiation.cc:407] Connection header received
0810 22:39:01.184009 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.184012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.184068 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.184144 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:01.185376 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.186235 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.187984 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.188402 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.188527 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.191445 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:01.191448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:01.191450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:01.191474 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:01.193005 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.193594 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.193597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.193598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.193668 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.194047 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.194050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.194051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.194242 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:01.194355 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.194847 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.194991 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 22:39:01.196024 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.182148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:41349 (local address 127.19.202.65:36341)
0810 22:39:01.182699 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.182761 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.183765 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.184310 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.184318 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.184669 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:01.185224 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.185237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.186387 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.186393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.187862 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.187874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.188541 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.188544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.188682 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.189350 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:01.189365 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:01.191266 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:01.193118 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:01.193123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:01.193138 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:01.193496 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:01.193764 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:01.193770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:01.193773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:01.193963 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:01.194354 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:01.194360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:01.194601 (+ 241us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.194806 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.195131 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":166,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:39:01.197386 21136 heartbeater.cc:347] Connected to a master server at 127.19.202.126:41349
I20260810 22:39:01.197685 21136 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:01.198323 21136 heartbeater.cc:511] Master 127.19.202.126:41349 requested a full tablet report, sending...
I20260810 22:39:01.200181 20911 ts_manager.cc:194] Registered new tserver with Master: c82543b337024922889b65506ba33d94 (127.19.202.65:44815)
I20260810 22:39:01.201361 20911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:36341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:01.213147 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41349
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:01.326912 21141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:01.327142 21141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:01.327183 21141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:01.331043 21141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:01.331091 21141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:01.331142 21141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:01.337502 21141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41349
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:01.338605 21141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:01.339648 21141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:01.349128 21151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:01.349261 21147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:01.349267 21148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:01.349320 21141 server_base.cc:1034] running on GCE node
I20260810 22:39:01.349897 21141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:01.350502 21141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:01.351676 21141 hybrid_clock.cc:648] HybridClock initialized: now 1786401541351662 us; error 29 us; skew 500 ppm
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:01.354420 21141 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:01.355269 21141 webserver.cc:460] Webserver started at http://127.19.202.66:35437/ using document root <none> and password file <none>
I20260810 22:39:01.355774 21141 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:01.355834 21141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:01.356009 21141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:01.358060 21141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "0c514684ab6e4e29bb8cd8981d79b496"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "9ab34b3e58dde6e90e7b53eb89a6ab9d"
server_key_iv: "98fe0fa9429ebe8e6c2f10a2880616b3"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.358579 21141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "0c514684ab6e4e29bb8cd8981d79b496"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "9ab34b3e58dde6e90e7b53eb89a6ab9d"
server_key_iv: "98fe0fa9429ebe8e6c2f10a2880616b3"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.362178 21141 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:01.364529 21161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.365773 21141 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:01.365865 21141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "0c514684ab6e4e29bb8cd8981d79b496"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "9ab34b3e58dde6e90e7b53eb89a6ab9d"
server_key_iv: "98fe0fa9429ebe8e6c2f10a2880616b3"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.365944 21141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:01.374567 21141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:01.377915 21141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:01.378079 21141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:01.378616 21141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:01.379503 21141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:01.379555 21141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.379595 21141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:01.379621 21141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.389254 21141 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:35233
I20260810 22:39:01.389278 21281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:35233 every 8 connection(s)
I20260810 22:39:01.390125 21141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:01.399493 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 21141
I20260810 22:39:01.399609 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:01.399968 20265 external_mini_cluster.cc:1428] Setting key b099611472f7ccc3245179c1a38c81b7
I20260810 22:39:01.403112 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.391986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:46035 (local address 127.19.202.126:41349)
0810 22:39:01.392122 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.392126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.392952 (+ 826us) server_negotiation.cc:407] Connection header received
0810 22:39:01.393733 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.393736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.393784 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.393868 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:01.394993 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.395522 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.396638 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.396935 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.397024 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.399354 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:01.399357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:01.399359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:01.399382 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:01.400872 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.401419 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.401423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.401425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.401514 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.401981 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.401985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.401986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.402182 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:01.402275 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.402876 (+ 601us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.402996 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:39:01.404102 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.392346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:41349 (local address 127.19.202.66:46035)
0810 22:39:01.392768 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.392828 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.393553 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.394021 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.394046 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.394368 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:01.394852 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.394866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.395635 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.395638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.396508 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.396517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.397065 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.397068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.397206 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.397727 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:01.397743 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:01.399150 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:01.400995 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:01.401000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:01.401010 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:01.401308 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:01.401614 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:01.401617 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:01.401619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:01.401808 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:01.402301 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:01.402310 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:01.402619 (+ 309us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.402825 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.403179 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":109,"threads_started":1}
I20260810 22:39:01.405253 21282 heartbeater.cc:347] Connected to a master server at 127.19.202.126:41349
I20260810 22:39:01.405505 21282 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:01.406090 21282 heartbeater.cc:511] Master 127.19.202.126:41349 requested a full tablet report, sending...
I20260810 22:39:01.407194 20911 ts_manager.cc:194] Registered new tserver with Master: 0c514684ab6e4e29bb8cd8981d79b496 (127.19.202.66:35233)
I20260810 22:39:01.407743 20911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:46035
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:01.428023 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41349
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:01.540751 21288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:01.540993 21288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:01.541035 21288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:01.544883 21288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:01.544926 21288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:01.544981 21288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:01.550058 21288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41349
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:01.550949 21288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:01.551905 21288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:01.561079 21294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:01.561126 21288 server_base.cc:1034] running on GCE node
W20260810 22:39:01.561156 21297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:01.561079 21293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:01.561744 21288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:01.562336 21288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:01.563521 21288 hybrid_clock.cc:648] HybridClock initialized: now 1786401541563506 us; error 25 us; skew 500 ppm
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:01.566147 21288 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:01.566941 21288 webserver.cc:460] Webserver started at http://127.19.202.67:35447/ using document root <none> and password file <none>
I20260810 22:39:01.567489 21288 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:01.567586 21288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:01.567827 21288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:01.570032 21288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "b86d19a517334119b8725d31d8852090"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "d1134d9cbb14b6d9cd811a0b641b4407"
server_key_iv: "0e82c9583574ae7bf4334ccec8139091"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.570574 21288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "b86d19a517334119b8725d31d8852090"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "d1134d9cbb14b6d9cd811a0b641b4407"
server_key_iv: "0e82c9583574ae7bf4334ccec8139091"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.574263 21288 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:39:01.576556 21305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.577827 21288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:01.577957 21288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "b86d19a517334119b8725d31d8852090"
format_stamp: "Formatted at 2026-08-10 22:39:01 on dist-test-slave-1x4g"
server_key: "d1134d9cbb14b6d9cd811a0b641b4407"
server_key_iv: "0e82c9583574ae7bf4334ccec8139091"
server_key_version: "encryptionkey@0"
I20260810 22:39:01.578063 21288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:01.593477 21288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:01.595933 21288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:01.596112 21288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:01.596665 21288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:01.597528 21288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:01.597596 21288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.597667 21288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:01.597708 21288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:01.606484 21288 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:33563
I20260810 22:39:01.606511 21434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:33563 every 8 connection(s)
I20260810 22:39:01.607322 21288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:01.614187 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 21288
I20260810 22:39:01.614328 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:01.614598 20265 external_mini_cluster.cc:1428] Setting key fb3967b6913e9cf3e7ab30214e316e2d
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:01.619899 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.609038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:34945 (local address 127.19.202.126:41349)
0810 22:39:01.609184 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.609189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.610046 (+ 857us) server_negotiation.cc:407] Connection header received
0810 22:39:01.610849 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.610852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.610899 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.610967 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:01.612025 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.612583 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.613721 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.613974 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.614091 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.616796 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:01.616800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:01.616803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:01.616829 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:01.618051 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.618525 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.618528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.618529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.618603 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.618906 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.618910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.618912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.619080 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:01.619178 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.619618 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.619734 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:39:01.620589 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.609443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:41349 (local address 127.19.202.67:34945)
0810 22:39:01.609872 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.609911 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.610646 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.611122 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.611128 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.611436 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:01.611885 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.611898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.612686 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.612689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.613617 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.613627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.614112 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.614115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.614244 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.614838 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:01.614871 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:01.616602 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:01.618164 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:01.618169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:01.618182 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:01.618421 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:01.618689 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:01.618692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:01.618694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:01.618818 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:01.619177 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:01.619184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:01.619407 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.619579 (+ 172us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.619866 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260810 22:39:01.621574 21436 heartbeater.cc:347] Connected to a master server at 127.19.202.126:41349
I20260810 22:39:01.621779 21436 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:01.622216 21436 heartbeater.cc:511] Master 127.19.202.126:41349 requested a full tablet report, sending...
I20260810 22:39:01.623135 20911 ts_manager.cc:194] Registered new tserver with Master: b86d19a517334119b8725d31d8852090 (127.19.202.67:33563)
I20260810 22:39:01.623663 20911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:34945
I20260810 22:39:01.628048 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:39:01.639364 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.632067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.19.202.126:41349)
0810 22:39:01.632203 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.632207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.632425 (+ 218us) server_negotiation.cc:407] Connection header received
0810 22:39:01.632490 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.632493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.632531 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.632601 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:01.633450 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.634053 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.634855 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.635080 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.635162 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.636421 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:01.636423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:01.636425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:01.636443 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:01.637674 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.638177 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.638181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.638184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.638265 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.638528 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.638531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.638533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.638749 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:01.638875 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.639105 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.639217 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:01.644843 20911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:01.648201 20911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:01.663926 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.658790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47578 (local address 127.19.202.67:33563)
0810 22:39:01.659496 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.659508 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.659529 (+ 21us) server_negotiation.cc:407] Connection header received
0810 22:39:01.659599 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.659602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.659774 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.659922 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:01.660134 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.661225 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.662665 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.663270 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.663387 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.663589 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.663695 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.663784 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":172,"threads_started":1}
I20260810 22:39:01.664034 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.657698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:35233 (local address 127.0.0.1:47210)
0810 22:39:01.658394 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.658435 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.658556 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.659677 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.659680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.659699 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:01.659880 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.659887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.661328 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.661340 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.662623 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.662633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.663606 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.663609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.663695 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.663721 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.663790 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.663859 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":85,"thread_start_us":107,"threads_started":1}
I20260810 22:39:01.664063 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.657963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44815 (local address 127.0.0.1:59758)
0810 22:39:01.658813 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.658827 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.658919 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.659661 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.659667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.659688 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:01.659883 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.659892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.660951 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.660956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.662392 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.662403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.663658 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.663661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.663802 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.663819 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.663874 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.663941 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":85,"spinlock_wait_cycles":896,"thread_start_us":117,"threads_started":1}
I20260810 22:39:01.663916 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.658582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:33563 (local address 127.0.0.1:47578)
0810 22:39:01.658965 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.658979 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.659061 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.659907 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.659911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.659931 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:01.660033 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.660039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.661374 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.661378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.662533 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.662544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.663391 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.663395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.663520 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.663558 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.663668 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.663780 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":57,"thread_start_us":84,"threads_started":1}
I20260810 22:39:01.664425 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.657961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.19.202.66:35233)
0810 22:39:01.659274 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.659290 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.659310 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:39:01.659383 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.659390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.659549 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.659689 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:01.660018 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.661182 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.662722 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.663484 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.663989 (+ 505us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.664138 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.664192 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.664281 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1220,"thread_start_us":124,"threads_started":1}
I20260810 22:39:01.664513 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.658477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.19.202.65:44815)
0810 22:39:01.658852 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.658869 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.658895 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:39:01.659099 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.659106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.659312 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.659468 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:01.660028 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.660769 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.662516 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.663363 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.663466 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.663610 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.664239 (+ 629us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.664326 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":79,"threads_started":1}
I20260810 22:39:01.667562 21360 tablet_service.cc:1467] Processing CreateTablet for tablet 90be2647905b4b9f9b03e2de52f5ce88 (DEFAULT_TABLE table=test-table [id=1e5b9790f696457ba0bac07d26df7561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:01.667587 21211 tablet_service.cc:1467] Processing CreateTablet for tablet 90be2647905b4b9f9b03e2de52f5ce88 (DEFAULT_TABLE table=test-table [id=1e5b9790f696457ba0bac07d26df7561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:01.668691 21360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90be2647905b4b9f9b03e2de52f5ce88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:01.668670 21211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90be2647905b4b9f9b03e2de52f5ce88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:01.669806 21063 tablet_service.cc:1467] Processing CreateTablet for tablet 90be2647905b4b9f9b03e2de52f5ce88 (DEFAULT_TABLE table=test-table [id=1e5b9790f696457ba0bac07d26df7561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:01.670817 21063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90be2647905b4b9f9b03e2de52f5ce88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:01.676472 21458 tablet_bootstrap.cc:492] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090: Bootstrap starting.
I20260810 22:39:01.677826 21459 tablet_bootstrap.cc:492] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496: Bootstrap starting.
I20260810 22:39:01.678527 21460 tablet_bootstrap.cc:492] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94: Bootstrap starting.
I20260810 22:39:01.679064 21458 tablet_bootstrap.cc:654] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:01.680238 21460 tablet_bootstrap.cc:654] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:01.680311 21459 tablet_bootstrap.cc:654] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:01.680527 21458 log.cc:824] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:01.681433 21460 log.cc:824] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:01.681516 21459 log.cc:824] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:01.682646 21458 tablet_bootstrap.cc:492] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090: No bootstrap required, opened a new log
I20260810 22:39:01.682837 21458 ts_tablet_manager.cc:1397] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 22:39:01.683209 21460 tablet_bootstrap.cc:492] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94: No bootstrap required, opened a new log
I20260810 22:39:01.683401 21460 ts_tablet_manager.cc:1397] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 22:39:01.686030 21459 tablet_bootstrap.cc:492] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496: No bootstrap required, opened a new log
I20260810 22:39:01.685942 21458 raft_consensus.cc:348] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.686193 21459 ts_tablet_manager.cc:1397] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260810 22:39:01.686228 21458 raft_consensus.cc:374] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:01.686275 21458 raft_consensus.cc:729] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86d19a517334119b8725d31d8852090, State: Initialized, Role: FOLLOWER
I20260810 22:39:01.686353 21460 raft_consensus.cc:348] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.686587 21460 raft_consensus.cc:374] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:01.686643 21460 raft_consensus.cc:729] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c82543b337024922889b65506ba33d94, State: Initialized, Role: FOLLOWER
I20260810 22:39:01.686721 21458 consensus_queue.cc:260] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.687100 21460 consensus_queue.cc:260] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.687559 21436 heartbeater.cc:503] Master 127.19.202.126:41349 was elected leader, sending a full tablet report...
I20260810 22:39:01.687772 21458 ts_tablet_manager.cc:1428] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090: Time spent starting tablet: real 0.005s user 0.003s sys 0.004s
I20260810 22:39:01.688017 21460 ts_tablet_manager.cc:1428] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:39:01.688319 21136 heartbeater.cc:503] Master 127.19.202.126:41349 was elected leader, sending a full tablet report...
I20260810 22:39:01.691183 21459 raft_consensus.cc:348] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.691414 21459 raft_consensus.cc:374] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:01.691476 21459 raft_consensus.cc:729] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c514684ab6e4e29bb8cd8981d79b496, State: Initialized, Role: FOLLOWER
I20260810 22:39:01.691913 21459 consensus_queue.cc:260] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.692535 21282 heartbeater.cc:503] Master 127.19.202.126:41349 was elected leader, sending a full tablet report...
I20260810 22:39:01.692937 21459 ts_tablet_manager.cc:1428] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260810 22:39:01.789187 21467 raft_consensus.cc:482] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:01.789441 21467 raft_consensus.cc:504] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.790634 21467 leader_election.cc:288] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c514684ab6e4e29bb8cd8981d79b496 (127.19.202.66:35233), c82543b337024922889b65506ba33d94 (127.19.202.65:44815)
I20260810 22:39:01.794409 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.791024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:60813 (local address 127.19.202.65:44815)
0810 22:39:01.791187 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.791191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.791209 (+ 18us) server_negotiation.cc:407] Connection header received
0810 22:39:01.791363 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.791365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.791409 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.791513 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:01.791814 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.792308 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.793491 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.793864 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.793991 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.794048 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.794225 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.794294 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:39:01.794410 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.790920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44815 (local address 127.19.202.67:60813)
0810 22:39:01.791131 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.791146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.791241 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.791525 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.791528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.791545 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:01.791670 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.791676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.792424 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.792430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.793362 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.793371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.793986 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.793989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.794092 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.794117 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.794206 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.794277 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":21}
I20260810 22:39:01.794612 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.790920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:35233 (local address 127.19.202.67:43541)
0810 22:39:01.791377 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:01.791388 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:01.791488 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:01.791770 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:01.791773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:01.791788 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:01.791921 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.791926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.792625 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.792630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.793516 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:01.793524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.794168 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.794171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:01.794294 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.794311 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 22:39:01.794400 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 22:39:01.794466 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":62,"thread_start_us":145,"threads_started":1}
I20260810 22:39:01.794606 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.791038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:43541 (local address 127.19.202.66:35233)
0810 22:39:01.791155 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.791160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.791469 (+ 309us) server_negotiation.cc:407] Connection header received
0810 22:39:01.791606 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.791609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.791651 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.791731 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:01.792037 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.792517 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.793639 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.794051 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.794171 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.794225 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.794417 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.794508 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:01.795306 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90be2647905b4b9f9b03e2de52f5ce88" candidate_uuid: "b86d19a517334119b8725d31d8852090" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c82543b337024922889b65506ba33d94" is_pre_election: true
I20260810 22:39:01.795639 21085 raft_consensus.cc:2454] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86d19a517334119b8725d31d8852090 in term 0.
I20260810 22:39:01.796125 21310 leader_election.cc:302] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b86d19a517334119b8725d31d8852090, c82543b337024922889b65506ba33d94; no voters:
I20260810 22:39:01.796397 21467 raft_consensus.cc:2781] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:01.796484 21467 raft_consensus.cc:482] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:01.796532 21467 raft_consensus.cc:3037] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:01.797479 21232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90be2647905b4b9f9b03e2de52f5ce88" candidate_uuid: "b86d19a517334119b8725d31d8852090" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c514684ab6e4e29bb8cd8981d79b496" is_pre_election: true
I20260810 22:39:01.797616 21467 raft_consensus.cc:504] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.797770 21232 raft_consensus.cc:2454] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86d19a517334119b8725d31d8852090 in term 0.
I20260810 22:39:01.798046 21467 leader_election.cc:288] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [CANDIDATE]: Term 1 election: Requested vote from peers 0c514684ab6e4e29bb8cd8981d79b496 (127.19.202.66:35233), c82543b337024922889b65506ba33d94 (127.19.202.65:44815)
I20260810 22:39:01.798372 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90be2647905b4b9f9b03e2de52f5ce88" candidate_uuid: "b86d19a517334119b8725d31d8852090" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c82543b337024922889b65506ba33d94"
I20260810 22:39:01.798501 21085 raft_consensus.cc:3037] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:01.798660 21232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90be2647905b4b9f9b03e2de52f5ce88" candidate_uuid: "b86d19a517334119b8725d31d8852090" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c514684ab6e4e29bb8cd8981d79b496"
I20260810 22:39:01.798784 21232 raft_consensus.cc:3037] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:01.799638 21085 raft_consensus.cc:2454] T 90be2647905b4b9f9b03e2de52f5ce88 P c82543b337024922889b65506ba33d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86d19a517334119b8725d31d8852090 in term 1.
I20260810 22:39:01.799911 21232 raft_consensus.cc:2454] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86d19a517334119b8725d31d8852090 in term 1.
I20260810 22:39:01.799960 21310 leader_election.cc:302] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b86d19a517334119b8725d31d8852090, c82543b337024922889b65506ba33d94; no voters:
I20260810 22:39:01.800123 21467 raft_consensus.cc:2781] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:01.800339 21467 raft_consensus.cc:686] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [term 1 LEADER]: Becoming Leader. State: Replica: b86d19a517334119b8725d31d8852090, State: Running, Role: LEADER
I20260810 22:39:01.800647 21467 consensus_queue.cc:237] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } }
I20260810 22:39:01.804072 20911 catalog_manager.cc:5495] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86d19a517334119b8725d31d8852090 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "b86d19a517334119b8725d31d8852090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c514684ab6e4e29bb8cd8981d79b496" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c82543b337024922889b65506ba33d94" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b86d19a517334119b8725d31d8852090" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 33563 } health_report { overall_health: HEALTHY } } }
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:01.851558 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.843136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.19.202.65:44815)
0810 22:39:01.843315 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.843319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.843404 (+ 85us) server_negotiation.cc:407] Connection header received
0810 22:39:01.843513 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.843515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.843556 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.843620 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:01.844273 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.844790 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.845608 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.845817 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.845897 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.848292 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:01.848303 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:01.848320 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:01.848341 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:01.849961 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.850366 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.850372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.850373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.850470 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.850695 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.850697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.850698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.850938 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:01.851093 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.851237 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.851355 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 22:39:01.858218 21437 tablet.cc:2369] T 90be2647905b4b9f9b03e2de52f5ce88 P b86d19a517334119b8725d31d8852090: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:01.879112 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.871153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.19.202.65:44815)
0810 22:39:01.871298 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.871303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.871440 (+ 137us) server_negotiation.cc:407] Connection header received
0810 22:39:01.871603 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.871606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.871655 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.871733 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:01.872426 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.873039 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.873858 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.874085 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.874183 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.876473 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:01.876476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:01.876478 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:01.876499 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:01.877788 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.878139 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.878142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.878143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.878214 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.878428 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.878430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.878432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.878581 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:01.878685 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.878890 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.878997 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260810 22:39:01.891319 21283 tablet.cc:2369] T 90be2647905b4b9f9b03e2de52f5ce88 P 0c514684ab6e4e29bb8cd8981d79b496: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:01.904585 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:01.896763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.19.202.65:44815)
0810 22:39:01.896919 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:01.896924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:01.897061 (+ 137us) server_negotiation.cc:407] Connection header received
0810 22:39:01.897212 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:01.897220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:01.897259 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:01.897338 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:01.897997 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.898478 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.899318 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:01.899532 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:01.899632 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:01.901968 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:01.901971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:01.901972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:01.901993 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:01.903177 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.903561 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.903563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.903564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.903631 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:01.903841 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:01.903843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:01.903844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:01.904020 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:01.904113 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:01.904316 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 22:39:01.904406 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 22:39:01.906261 21041 tablet_service.cc:2908] Rejecting scan request for tablet 90be2647905b4b9f9b03e2de52f5ce88: Uninitialized: clean time has not yet been initialized
I20260810 22:39:01.908542 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 20984
I20260810 22:39:01.920182 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 21141
I20260810 22:39:01.927544 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 21288
I20260810 22:39:01.934855 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 20875
2026-08-10T22:39:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2427 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[21487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[21487](info): set up 2 sockets
Aug 10 22:39:01 dist-test-slave-1x4g krb5kdc[21487](info): commencing operation
krb5kdc: starting...
W20260810 22:39:03.000993 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:03.115631 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:45525
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:45525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:03.220319 21500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:03.220567 21500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:03.220642 21500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:03.224309 21500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:03.224383 21500 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 22:39:03.224421 21500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:03.224478 21500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:03.224514 21500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:03.224545 21500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:03.229338 21500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:45525
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:45525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:03.230259 21500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:03.231158 21500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:03.239786 21506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:03.239796 21509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:03.239777 21507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:03.239912 21500 server_base.cc:1034] running on GCE node
I20260810 22:39:03.240677 21500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:03.241746 21500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:03.242944 21500 hybrid_clock.cc:648] HybridClock initialized: now 1786401543242927 us; error 29 us; skew 500 ppm
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:03.250468 21500 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:03.251302 21500 webserver.cc:460] Webserver started at http://127.19.202.126:34869/ using document root <none> and password file <none>
I20260810 22:39:03.251840 21500 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:03.251982 21500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:03.252216 21500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:03.254324 21500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "56c8aae4a2824a92864bd01a83a12c6c"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "3665dbc5b7188e22ee5eabae2f39a703"
server_key_iv: "76ffdbfa2db4284e32c3b1ad3e7f901a"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.254890 21500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "56c8aae4a2824a92864bd01a83a12c6c"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "3665dbc5b7188e22ee5eabae2f39a703"
server_key_iv: "76ffdbfa2db4284e32c3b1ad3e7f901a"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.258471 21500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 22:39:03.260757 21517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:03.261982 21500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:03.262118 21500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "56c8aae4a2824a92864bd01a83a12c6c"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "3665dbc5b7188e22ee5eabae2f39a703"
server_key_iv: "76ffdbfa2db4284e32c3b1ad3e7f901a"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.262225 21500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:03.284554 21500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:03.287690 21500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:03.287881 21500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:03.296021 21500 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:45525
I20260810 22:39:03.296054 21585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:45525 every 8 connection(s)
I20260810 22:39:03.296949 21500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:03.299688 21586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:03.301488 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 21500
I20260810 22:39:03.301587 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:03.301852 20265 external_mini_cluster.cc:1428] Setting key 1c4ff1ef9d32a408c474818405138d29
I20260810 22:39:03.305879 21586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c: Bootstrap starting.
I20260810 22:39:03.308019 21586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:03.308727 21586 log.cc:824] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c: Log is configured to *not* fsync() on all Append() calls
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:03.310824 21586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c: No bootstrap required, opened a new log
I20260810 22:39:03.314239 21586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c8aae4a2824a92864bd01a83a12c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45525 } }
I20260810 22:39:03.314481 21586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:03.314561 21586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c8aae4a2824a92864bd01a83a12c6c, State: Initialized, Role: FOLLOWER
I20260810 22:39:03.315014 21586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c8aae4a2824a92864bd01a83a12c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45525 } }
I20260810 22:39:03.315184 21586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:03.315253 21586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:03.315375 21586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:03.316236 21586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c8aae4a2824a92864bd01a83a12c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45525 } }
I20260810 22:39:03.316489 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.303215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51084 (local address 127.19.202.126:45525)
0810 22:39:03.303644 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:03.303659 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:03.303693 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:03.304354 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:03.304371 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:03.304669 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:03.305040 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:03.305824 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.306826 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.307679 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.308166 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.308364 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.310998 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:03.311003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:03.311015 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:03.311040 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:03.313423 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.313899 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.313903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.313907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.314006 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.314282 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.314284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.314285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.314761 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:03.314964 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:03.315278 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 22:39:03.315741 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":131,"threads_started":1}
I20260810 22:39:03.316612 21586 leader_election.cc:302] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56c8aae4a2824a92864bd01a83a12c6c; no voters:
I20260810 22:39:03.316991 21586 leader_election.cc:288] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:03.317060 21592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:03.317281 21592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [term 1 LEADER]: Becoming Leader. State: Replica: 56c8aae4a2824a92864bd01a83a12c6c, State: Running, Role: LEADER
I20260810 22:39:03.317588 21592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c8aae4a2824a92864bd01a83a12c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45525 } }
I20260810 22:39:03.318751 21592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56c8aae4a2824a92864bd01a83a12c6c. Latest consensus state: current_term: 1 leader_uuid: "56c8aae4a2824a92864bd01a83a12c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c8aae4a2824a92864bd01a83a12c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45525 } } }
I20260810 22:39:03.318868 21592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:03.319211 21593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56c8aae4a2824a92864bd01a83a12c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c8aae4a2824a92864bd01a83a12c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45525 } } }
I20260810 22:39:03.319320 21593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:03.319366 21586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:03.319589 21596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:03.322559 21596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:03.328133 21596 catalog_manager.cc:1347] Generated new cluster ID: 1af6d92b2cdc49b6b71d1a9e25326545
I20260810 22:39:03.328197 21596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:03.341907 21596 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:03.343310 21596 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:03.350915 21596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c: Generated new TSK 0
I20260810 22:39:03.351608 21596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:03.387140 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:45525
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:03.517635 21615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:03.517872 21615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:03.517915 21615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:03.522002 21615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:03.522049 21615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:03.522086 21615 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:03.522117 21615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:03.527153 21615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:45525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:03.528116 21615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:03.529089 21615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:03.538430 21621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:03.538461 21624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:03.538612 21615 server_base.cc:1034] running on GCE node
W20260810 22:39:03.538462 21622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:03.539152 21615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:03.539902 21615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:03.541083 21615 hybrid_clock.cc:648] HybridClock initialized: now 1786401543541065 us; error 34 us; skew 500 ppm
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:03.543931 21615 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:03.544821 21615 webserver.cc:460] Webserver started at http://127.19.202.65:42351/ using document root <none> and password file <none>
I20260810 22:39:03.545392 21615 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:03.545456 21615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:03.545688 21615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:03.547691 21615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "bea2c5f285804f028de79c0868d03696"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "17f1eafd82f687c5672c1afe1f40201e"
server_key_iv: "a89860ad3460384c7dcd533cddf4de9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.548269 21615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "bea2c5f285804f028de79c0868d03696"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "17f1eafd82f687c5672c1afe1f40201e"
server_key_iv: "a89860ad3460384c7dcd533cddf4de9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.552101 21615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 22:39:03.554550 21633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:03.555799 21615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 22:39:03.555951 21615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "bea2c5f285804f028de79c0868d03696"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "17f1eafd82f687c5672c1afe1f40201e"
server_key_iv: "a89860ad3460384c7dcd533cddf4de9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.556066 21615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:03.573014 21615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:03.575855 21615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:03.576088 21615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:03.577575 21615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:03.577651 21615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:03.577723 21615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:03.577768 21615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:03.589215 21615 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:43415
I20260810 22:39:03.589329 21757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:43415 every 8 connection(s)
I20260810 22:39:03.590417 21615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:03.591730 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.578444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:34393 (local address 127.19.202.126:45525)
0810 22:39:03.578574 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:03.578578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:03.579605 (+ 1027us) server_negotiation.cc:407] Connection header received
0810 22:39:03.580572 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:03.580578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:03.580650 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:03.580752 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:03.582100 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.583041 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.584304 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.584646 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.584772 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.587657 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:03.587664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:03.587670 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:03.587707 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:03.589441 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.590038 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.590041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.590042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.590122 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.590544 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.590547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.590549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.590748 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:03.590862 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:03.591467 (+ 605us) server_negotiation.cc:299] Negotiation successful
0810 22:39:03.591610 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:39:03.592447 21642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.578979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:45525 (local address 127.19.202.65:34393)
0810 22:39:03.579460 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:03.579502 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:03.580311 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:03.581098 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:03.581107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:03.581468 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:03.581941 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.581954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.583186 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.583191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.584128 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.584138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.584844 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.584849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.585071 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.585753 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:03.585774 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:03.587497 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:03.589577 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.589582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.589595 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.589917 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.590226 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.590229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.590231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.590432 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.590863 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:03.590871 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:03.591176 (+ 305us) client_negotiation.cc:769] Sending connection context
0810 22:39:03.591380 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 22:39:03.591645 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":138,"threads_started":1}
I20260810 22:39:03.593778 21758 heartbeater.cc:347] Connected to a master server at 127.19.202.126:45525
I20260810 22:39:03.593940 21758 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:03.594498 21758 heartbeater.cc:511] Master 127.19.202.126:45525 requested a full tablet report, sending...
I20260810 22:39:03.594537 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 21615
I20260810 22:39:03.594678 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:03.594992 20265 external_mini_cluster.cc:1428] Setting key 3ddbc0d7a8dcadef4d0630d4356a0a34
I20260810 22:39:03.596154 21536 ts_manager.cc:194] Registered new tserver with Master: bea2c5f285804f028de79c0868d03696 (127.19.202.65:43415)
I20260810 22:39:03.597690 21536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:34393
I20260810 22:39:03.603824 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.596166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:45045 (local address 127.19.202.126:45525)
0810 22:39:03.596299 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:03.596303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:03.596498 (+ 195us) server_negotiation.cc:407] Connection header received
0810 22:39:03.596626 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:03.596628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:03.596665 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:03.596729 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:03.597342 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.598049 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.599006 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.599217 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.599324 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.600728 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:03.600731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:03.600733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:03.600753 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:03.602245 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.602669 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.602673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.602675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.602755 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.603037 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.603041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.603043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.603204 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:03.603310 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:03.603568 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 22:39:03.603672 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:03.603840 21766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.596106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:45525 (local address 127.19.202.65:45045)
0810 22:39:03.596425 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:03.596442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:03.596534 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:03.596770 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:03.596796 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:03.597010 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:03.597126 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.597133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.598195 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.598200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.598871 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.598881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.599330 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.599334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.599433 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.600104 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:03.600125 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:03.600593 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:03.602359 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.602364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.602366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.602568 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.602847 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.602850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.602852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.602945 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.603301 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:03.603304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:03.603404 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 22:39:03.603513 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 22:39:03.603653 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:03.623291 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:45525
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:03.733949 21769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:03.734185 21769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:03.734226 21769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:03.738499 21769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:03.738550 21769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:03.738615 21769 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:03.738670 21769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:03.743855 21769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:45525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:03.744882 21769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:03.745877 21769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:03.755256 21775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:03.755280 21778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:03.755306 21774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:03.755589 21769 server_base.cc:1034] running on GCE node
I20260810 22:39:03.755998 21769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:03.756659 21769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:03.757822 21769 hybrid_clock.cc:648] HybridClock initialized: now 1786401543757788 us; error 48 us; skew 500 ppm
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:03.760684 21769 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:03.761494 21769 webserver.cc:460] Webserver started at http://127.19.202.66:44047/ using document root <none> and password file <none>
I20260810 22:39:03.762061 21769 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:03.762122 21769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:03.762349 21769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:03.764264 21769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "603dd7c7197243b2a09bf060901a3587"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "4feef68d4b45946eb5c47d0cd3498769"
server_key_iv: "2a31383c4c0df932f264adadb3540c03"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.764870 21769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "603dd7c7197243b2a09bf060901a3587"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "4feef68d4b45946eb5c47d0cd3498769"
server_key_iv: "2a31383c4c0df932f264adadb3540c03"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.768555 21769 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:39:03.770876 21785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:03.772050 21769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:03.772166 21769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "603dd7c7197243b2a09bf060901a3587"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "4feef68d4b45946eb5c47d0cd3498769"
server_key_iv: "2a31383c4c0df932f264adadb3540c03"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.772264 21769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:03.788662 21769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:03.791487 21769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:03.791666 21769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:03.793043 21769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:03.793102 21769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:03.793159 21769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:03.793190 21769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:03.803704 21769 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:37693
I20260810 22:39:03.803742 21912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:37693 every 8 connection(s)
I20260810 22:39:03.804509 21769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:03.807459 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.793848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:44797 (local address 127.19.202.126:45525)
0810 22:39:03.793988 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:03.793991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:03.794993 (+ 1002us) server_negotiation.cc:407] Connection header received
0810 22:39:03.796014 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:03.796017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:03.796080 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:03.796147 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:03.797978 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.798821 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.800675 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.800959 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.801043 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.803973 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:03.803976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:03.803978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:03.804001 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:03.805684 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.806143 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.806145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.806146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.806223 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.806537 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.806539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.806541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.806699 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:03.806771 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:03.807244 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 22:39:03.807336 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:03.808200 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.794369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:45525 (local address 127.19.202.66:44797)
0810 22:39:03.794828 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:03.794881 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:03.795762 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:03.796631 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:03.796642 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:03.797117 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:03.797819 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.797853 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.798971 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.798977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.800532 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.800546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.801136 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.801143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.801350 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.801948 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:03.801975 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:03.803759 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:03.805793 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.805798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.805808 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.806059 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.806310 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.806312 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.806313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.806454 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.806785 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:03.806790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:03.807036 (+ 246us) client_negotiation.cc:769] Sending connection context
0810 22:39:03.807195 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 22:39:03.807428 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":140,"threads_started":1}
I20260810 22:39:03.809337 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 21769
I20260810 22:39:03.809382 21913 heartbeater.cc:347] Connected to a master server at 127.19.202.126:45525
I20260810 22:39:03.809463 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:03.809537 21913 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:03.809784 20265 external_mini_cluster.cc:1428] Setting key 65c4dca7616fbe449fee5726f963ad43
I20260810 22:39:03.810020 21913 heartbeater.cc:511] Master 127.19.202.126:45525 requested a full tablet report, sending...
I20260810 22:39:03.811054 21536 ts_manager.cc:194] Registered new tserver with Master: 603dd7c7197243b2a09bf060901a3587 (127.19.202.66:37693)
I20260810 22:39:03.811764 21536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:44797
I20260810 22:39:03.821311 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.811683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:40665 (local address 127.19.202.126:45525)
0810 22:39:03.813515 (+ 1832us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:03.813519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:03.813533 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:39:03.813565 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:03.813568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:03.813626 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:03.813725 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:03.814372 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.815115 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.816331 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.816540 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.816635 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.817865 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:03.817868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:03.817870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:03.817909 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:03.819459 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.819933 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.819935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.819936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.820005 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:03.820312 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:03.820315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:03.820316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:03.820573 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:03.820649 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:03.821016 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 22:39:03.821118 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1740}
I20260810 22:39:03.821321 21921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:03.811695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:45525 (local address 127.19.202.66:40665)
0810 22:39:03.811975 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:03.811990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:03.812075 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:03.813758 (+ 1683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:03.813761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:03.814105 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:03.814232 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.814239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.815278 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.815281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.816188 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:03.816203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:03.816667 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:03.816674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:03.816824 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:03.817274 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:03.817293 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:03.817736 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:03.819572 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.819575 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.819577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.819826 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.820104 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:03.820108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:03.820111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:03.820231 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:03.820672 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:03.820678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:03.820798 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 22:39:03.820950 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 22:39:03.821101 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":198,"threads_started":1}
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:03.837641 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:45525
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:03.947024 21925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:03.947252 21925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:03.947290 21925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:03.951043 21925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:03.951086 21925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:03.951150 21925 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:03.951225 21925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:03.956259 21925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:45525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:03.957208 21925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:03.958151 21925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:03.967736 21930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:03.967706 21932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:03.967721 21925 server_base.cc:1034] running on GCE node
W20260810 22:39:03.967746 21934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:03.968417 21925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:03.969139 21925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:03.970306 21925 hybrid_clock.cc:648] HybridClock initialized: now 1786401543970275 us; error 43 us; skew 500 ppm
Aug 10 22:39:03 dist-test-slave-1x4g krb5kdc[21487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:03.973272 21925 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:03.974072 21925 webserver.cc:460] Webserver started at http://127.19.202.67:42933/ using document root <none> and password file <none>
I20260810 22:39:03.974587 21925 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:03.974678 21925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:03.974928 21925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:03.976985 21925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "af76116db6354c818e2165d0e6ea3f60"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "2c6cd684ee4aac8e8891421378a9c60f"
server_key_iv: "d8d7455ee42cc7ac077b6f3d668cc459"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.977583 21925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "af76116db6354c818e2165d0e6ea3f60"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "2c6cd684ee4aac8e8891421378a9c60f"
server_key_iv: "d8d7455ee42cc7ac077b6f3d668cc459"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.981174 21925 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:03.983598 21941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:03.984731 21925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:03.984896 21925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "af76116db6354c818e2165d0e6ea3f60"
format_stamp: "Formatted at 2026-08-10 22:39:03 on dist-test-slave-1x4g"
server_key: "2c6cd684ee4aac8e8891421378a9c60f"
server_key_iv: "d8d7455ee42cc7ac077b6f3d668cc459"
server_key_version: "encryptionkey@0"
I20260810 22:39:03.985010 21925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:04.004668 21925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:04.008126 21925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:04.008342 21925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:04.010032 21925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:04.010110 21925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:04.010221 21925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:04.010275 21925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:04 dist-test-slave-1x4g krb5kdc[21487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:04.021596 21925 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:46783
I20260810 22:39:04.021639 22066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:46783 every 8 connection(s)
I20260810 22:39:04.022709 21925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:04.023604 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 21925
I20260810 22:39:04.023741 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:04.024010 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.010676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:50983 (local address 127.19.202.126:45525)
0810 22:39:04.010831 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.010836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.011737 (+ 901us) server_negotiation.cc:407] Connection header received
0810 22:39:04.012736 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.012740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.012827 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.012944 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:04.014452 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.014954 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.016640 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.017075 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.017218 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.019810 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:04.019814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:04.019817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:04.019842 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:04.021788 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.022282 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.022289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.022294 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.022412 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.022756 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.022762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.022765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.023012 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:04.023137 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.023761 (+ 624us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.023888 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:39:04.024044 20265 external_mini_cluster.cc:1428] Setting key 0646fcaec46086a4a2bb68395283ec25
I20260810 22:39:04.024999 21951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.011111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:45525 (local address 127.19.202.67:50983)
0810 22:39:04.011561 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.011611 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.012490 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.013148 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.013156 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.013651 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:04.014242 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.014256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.015078 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.015081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.016448 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.016458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.017265 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.017269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.017490 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.018069 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:04.018089 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:04.019648 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:04.021924 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:04.021929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:04.021939 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:04.022174 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:04.022517 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:04.022519 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:04.022520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:04.022652 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:04.023160 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:04.023166 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:04.023478 (+ 312us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.023692 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.023996 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":130,"threads_started":1}
I20260810 22:39:04.026207 22067 heartbeater.cc:347] Connected to a master server at 127.19.202.126:45525
I20260810 22:39:04.026374 22067 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:04.026821 22067 heartbeater.cc:511] Master 127.19.202.126:45525 requested a full tablet report, sending...
I20260810 22:39:04.027874 21536 ts_manager.cc:194] Registered new tserver with Master: af76116db6354c818e2165d0e6ea3f60 (127.19.202.67:46783)
I20260810 22:39:04.028627 21536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:50983
I20260810 22:39:04.034857 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.028436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:32947 (local address 127.19.202.126:45525)
0810 22:39:04.028537 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.028541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.028833 (+ 292us) server_negotiation.cc:407] Connection header received
0810 22:39:04.028956 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.028959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.029000 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.029085 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:04.029498 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.029982 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.030767 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.030993 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.031096 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.032057 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:04.032061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:04.032063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:04.032086 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:04.033314 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.033728 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.033731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.033733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.033811 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.034085 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.034089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.034091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.034251 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:04.034356 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.034628 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.034731 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:39:04.034864 22073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.028462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:45525 (local address 127.19.202.67:32947)
0810 22:39:04.028717 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.028732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.028851 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.029101 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.029103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.029298 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:04.029383 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.029389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.030102 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.030106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.030648 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.030655 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.031100 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.031104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.031199 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.031621 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:04.031631 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:04.031899 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:04.033433 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:04.033436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:04.033438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:04.033627 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:04.033904 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:04.033907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:04.033909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:04.033994 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:04.034348 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:04.034352 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:04.034452 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.034559 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.034698 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":105,"threads_started":1}
I20260810 22:39:04.039935 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:39:04 dist-test-slave-1x4g krb5kdc[21487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:39:04 dist-test-slave-1x4g krb5kdc[21487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:04.065990 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.057772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.19.202.126:45525)
0810 22:39:04.057962 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.057969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.057989 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:39:04.058041 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.058043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.058083 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.058159 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:04.058795 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.059410 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.060184 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.060391 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.060476 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.062804 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:04.062807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:04.062808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:04.062829 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:04.064333 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.064710 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.064712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.064713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.064818 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.065156 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.065159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.065161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.065340 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:04.065475 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.065724 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.065828 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 22:39:04.068519 21536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:04.071640 21536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:04.085630 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.079756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:37693 (local address 127.0.0.1:53714)
0810 22:39:04.080075 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.080126 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.080249 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.080990 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.080996 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.081020 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:04.081149 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.081154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.082722 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.082724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.083960 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.083972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.085076 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.085080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.085181 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.085217 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.085323 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.085468 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260810 22:39:04.085923 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.080828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48928 (local address 127.19.202.65:43415)
0810 22:39:04.081110 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.081127 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.081150 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:39:04.081227 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.081234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.081427 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.081576 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:04.082017 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.083004 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.084290 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.085106 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.085200 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.085514 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.085585 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.085675 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":118,"threads_started":1}
I20260810 22:39:04.085978 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.079959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53714 (local address 127.19.202.66:37693)
0810 22:39:04.080390 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.080404 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.080428 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:39:04.080517 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.080526 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.080739 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.080993 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:04.081300 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.082609 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.084101 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.084966 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.085077 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.085240 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.085726 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.085826 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":125,"threads_started":1}
I20260810 22:39:04.086275 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.080473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:46783 (local address 127.0.0.1:40372)
0810 22:39:04.080697 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.080712 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.080846 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.082251 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.082254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.082273 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:04.082365 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.082371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.083561 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.083564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.084720 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.084730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.085825 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.085828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.085981 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.086014 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.086077 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.086141 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":24,"thread_start_us":86,"threads_started":1}
I20260810 22:39:04.085647 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.080115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:43415 (local address 127.0.0.1:48928)
0810 22:39:04.080481 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.080493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.080567 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.081730 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.081736 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.081756 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:04.081871 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.081882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.083125 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.083129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.084147 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.084161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.085316 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.085320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.085398 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.085418 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.085495 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.085560 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":26,"thread_start_us":100,"threads_started":1}
I20260810 22:39:04.086824 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.081324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40372 (local address 127.19.202.67:46783)
0810 22:39:04.081810 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.081830 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.081860 (+ 30us) server_negotiation.cc:407] Connection header received
0810 22:39:04.081933 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.081937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.082094 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.082250 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:04.082482 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.083440 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.084883 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.085693 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.086437 (+ 744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.086559 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.086595 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.086653 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":183,"threads_started":1}
I20260810 22:39:04.088982 21843 tablet_service.cc:1467] Processing CreateTablet for tablet c819e90cf20b476b82ad3217659fbe3d (DEFAULT_TABLE table=test-table [id=a06d3ee5174e4541b5e40e2342405b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:04.089380 21994 tablet_service.cc:1467] Processing CreateTablet for tablet c819e90cf20b476b82ad3217659fbe3d (DEFAULT_TABLE table=test-table [id=a06d3ee5174e4541b5e40e2342405b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:04.089380 21687 tablet_service.cc:1467] Processing CreateTablet for tablet c819e90cf20b476b82ad3217659fbe3d (DEFAULT_TABLE table=test-table [id=a06d3ee5174e4541b5e40e2342405b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:04.090029 21843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c819e90cf20b476b82ad3217659fbe3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.090344 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c819e90cf20b476b82ad3217659fbe3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.090351 21994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c819e90cf20b476b82ad3217659fbe3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.097759 22090 tablet_bootstrap.cc:492] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Bootstrap starting.
I20260810 22:39:04.098181 22091 tablet_bootstrap.cc:492] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Bootstrap starting.
I20260810 22:39:04.098647 22092 tablet_bootstrap.cc:492] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Bootstrap starting.
I20260810 22:39:04.100533 22091 tablet_bootstrap.cc:654] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.100848 22092 tablet_bootstrap.cc:654] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.100848 22090 tablet_bootstrap.cc:654] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.101526 22091 log.cc:824] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:04.101720 22090 log.cc:824] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:04.101720 22092 log.cc:824] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:04.103533 22090 tablet_bootstrap.cc:492] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: No bootstrap required, opened a new log
I20260810 22:39:04.103653 22091 tablet_bootstrap.cc:492] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: No bootstrap required, opened a new log
I20260810 22:39:04.103659 22092 tablet_bootstrap.cc:492] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: No bootstrap required, opened a new log
I20260810 22:39:04.103739 22090 ts_tablet_manager.cc:1397] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:39:04.103828 22092 ts_tablet_manager.cc:1397] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:39:04.103829 22091 ts_tablet_manager.cc:1397] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:39:04.106755 22090 raft_consensus.cc:348] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.107040 22090 raft_consensus.cc:374] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.107133 22090 raft_consensus.cc:729] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603dd7c7197243b2a09bf060901a3587, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.107581 22090 consensus_queue.cc:260] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.107676 22092 raft_consensus.cc:348] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.107676 22091 raft_consensus.cc:348] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.107909 22091 raft_consensus.cc:374] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.107992 22092 raft_consensus.cc:374] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.108006 22091 raft_consensus.cc:729] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea2c5f285804f028de79c0868d03696, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.108098 22092 raft_consensus.cc:729] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af76116db6354c818e2165d0e6ea3f60, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.108489 22092 consensus_queue.cc:260] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.108486 22091 consensus_queue.cc:260] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.108592 22090 ts_tablet_manager.cc:1428] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:39:04.108968 21913 heartbeater.cc:503] Master 127.19.202.126:45525 was elected leader, sending a full tablet report...
I20260810 22:39:04.109443 22092 ts_tablet_manager.cc:1428] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:39:04.109454 22091 ts_tablet_manager.cc:1428] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 22:39:04.109776 21758 heartbeater.cc:503] Master 127.19.202.126:45525 was elected leader, sending a full tablet report...
I20260810 22:39:04.109786 22067 heartbeater.cc:503] Master 127.19.202.126:45525 was elected leader, sending a full tablet report...
W20260810 22:39:04.273993 22068 tablet.cc:2369] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:39:04.306430 21915 tablet.cc:2369] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:39:04.342144 21759 tablet.cc:2369] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:39:04.401820 22099 raft_consensus.cc:482] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:04.402079 22099 raft_consensus.cc:504] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.403287 22099 leader_election.cc:288] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 603dd7c7197243b2a09bf060901a3587 (127.19.202.66:37693), bea2c5f285804f028de79c0868d03696 (127.19.202.65:43415)
I20260810 22:39:04.407596 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.403668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:43355 (local address 127.19.202.66:37693)
0810 22:39:04.403801 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.403806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.403823 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:39:04.403882 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.403885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.403958 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.404055 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:04.404363 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.405230 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.406463 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.407081 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.407167 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.407209 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.407440 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.407499 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:39:04.407612 21951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.403511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:37693 (local address 127.19.202.67:43355)
0810 22:39:04.403697 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.403713 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.403803 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.404082 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.404085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.404102 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:04.404249 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.404256 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.405348 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.405352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.406333 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.406342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.407186 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.407190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.407315 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.407339 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.407416 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.407485 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 22:39:04.408298 22100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.403539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:43415 (local address 127.19.202.67:52135)
0810 22:39:04.403889 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.403900 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.403996 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.404257 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.404259 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.404284 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:04.405041 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.405052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.405971 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.405974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.406941 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.406951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.407910 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.407922 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.408026 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.408052 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.408124 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.408180 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":114,"threads_started":1}
I20260810 22:39:04.408317 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.403637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:52135 (local address 127.19.202.65:43415)
0810 22:39:04.403770 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.403774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.403982 (+ 208us) server_negotiation.cc:407] Connection header received
0810 22:39:04.404103 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.404106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.404148 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.404237 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:04.405159 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.405858 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.407061 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.407631 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.407759 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.407837 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.408169 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.408233 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:39:04.408273 21864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c819e90cf20b476b82ad3217659fbe3d" candidate_uuid: "af76116db6354c818e2165d0e6ea3f60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603dd7c7197243b2a09bf060901a3587" is_pre_election: true
I20260810 22:39:04.408605 21864 raft_consensus.cc:2454] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af76116db6354c818e2165d0e6ea3f60 in term 0.
I20260810 22:39:04.409045 21709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c819e90cf20b476b82ad3217659fbe3d" candidate_uuid: "af76116db6354c818e2165d0e6ea3f60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea2c5f285804f028de79c0868d03696" is_pre_election: true
I20260810 22:39:04.409201 21944 leader_election.cc:302] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603dd7c7197243b2a09bf060901a3587, af76116db6354c818e2165d0e6ea3f60; no voters:
I20260810 22:39:04.409386 21709 raft_consensus.cc:2454] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af76116db6354c818e2165d0e6ea3f60 in term 0.
I20260810 22:39:04.409509 22099 raft_consensus.cc:2781] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:04.409614 22099 raft_consensus.cc:482] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:04.409667 22099 raft_consensus.cc:3037] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:04.410748 22099 raft_consensus.cc:504] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.411046 22099 leader_election.cc:288] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [CANDIDATE]: Term 1 election: Requested vote from peers 603dd7c7197243b2a09bf060901a3587 (127.19.202.66:37693), bea2c5f285804f028de79c0868d03696 (127.19.202.65:43415)
I20260810 22:39:04.411330 21864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c819e90cf20b476b82ad3217659fbe3d" candidate_uuid: "af76116db6354c818e2165d0e6ea3f60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603dd7c7197243b2a09bf060901a3587"
I20260810 22:39:04.411360 21709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c819e90cf20b476b82ad3217659fbe3d" candidate_uuid: "af76116db6354c818e2165d0e6ea3f60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea2c5f285804f028de79c0868d03696"
I20260810 22:39:04.411468 21709 raft_consensus.cc:3037] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:04.411458 21864 raft_consensus.cc:3037] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:04.412735 21709 raft_consensus.cc:2454] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af76116db6354c818e2165d0e6ea3f60 in term 1.
I20260810 22:39:04.412735 21864 raft_consensus.cc:2454] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af76116db6354c818e2165d0e6ea3f60 in term 1.
I20260810 22:39:04.413116 21943 leader_election.cc:302] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af76116db6354c818e2165d0e6ea3f60, bea2c5f285804f028de79c0868d03696; no voters:
I20260810 22:39:04.413321 22099 raft_consensus.cc:2781] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:04.413472 22099 raft_consensus.cc:686] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 1 LEADER]: Becoming Leader. State: Replica: af76116db6354c818e2165d0e6ea3f60, State: Running, Role: LEADER
I20260810 22:39:04.413784 22099 consensus_queue.cc:237] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } }
I20260810 22:39:04.416867 21535 catalog_manager.cc:5495] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 reported cstate change: term changed from 0 to 1, leader changed from <none> to af76116db6354c818e2165d0e6ea3f60 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "af76116db6354c818e2165d0e6ea3f60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:04.450532 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.447303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40384 (local address 127.19.202.67:46783)
0810 22:39:04.447423 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.447427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.447441 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:39:04.447625 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.447627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.447674 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.447749 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:04.448000 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.448490 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.449311 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.449507 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.449578 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.449834 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:04.450221 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:04.450301 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.450391 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.450440 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:39:04.456156 21864 raft_consensus.cc:1261] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Refusing update from remote peer af76116db6354c818e2165d0e6ea3f60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:04.456810 22101 consensus_queue.cc:1035] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:04.458366 21709 raft_consensus.cc:1261] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Refusing update from remote peer af76116db6354c818e2165d0e6ea3f60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:04.458959 22099 consensus_queue.cc:1035] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:04.465162 22104 mvcc.cc:205] Tried to move back new op lower bound from 7317100726084857856 to 7317100725921452032. Current Snapshot: MvccSnapshot[applied={T|T < 7317100726084857856}]
I20260810 22:39:04.465535 22105 mvcc.cc:205] Tried to move back new op lower bound from 7317100726084857856 to 7317100725921452032. Current Snapshot: MvccSnapshot[applied={T|T < 7317100726084857856}]
I20260810 22:39:04.467140 22107 mvcc.cc:205] Tried to move back new op lower bound from 7317100726084857856 to 7317100725921452032. Current Snapshot: MvccSnapshot[applied={T|T < 7317100726084857856}]
I20260810 22:39:04.479020 21535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:04.479219 21535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:04.481751 21535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c: Sending DeleteTablet for 3 replicas of tablet c819e90cf20b476b82ad3217659fbe3d
I20260810 22:39:04.482610 21687 tablet_service.cc:1514] Processing DeleteTablet for tablet c819e90cf20b476b82ad3217659fbe3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:48928
I20260810 22:39:04.482879 21843 tablet_service.cc:1514] Processing DeleteTablet for tablet c819e90cf20b476b82ad3217659fbe3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:53714
I20260810 22:39:04.483220 22113 tablet_replica.cc:330] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: stopping tablet replica
I20260810 22:39:04.483220 21994 tablet_service.cc:1514] Processing DeleteTablet for tablet c819e90cf20b476b82ad3217659fbe3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:40372
I20260810 22:39:04.483582 22113 raft_consensus.cc:2229] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:04.483829 22114 tablet_replica.cc:330] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: stopping tablet replica
I20260810 22:39:04.483880 22113 raft_consensus.cc:2258] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:04.484229 22114 raft_consensus.cc:2229] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:04.484381 22115 tablet_replica.cc:330] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: stopping tablet replica
I20260810 22:39:04.484556 21535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 22:39:04.484712 22115 raft_consensus.cc:2229] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:04.485111 22115 raft_consensus.cc:2258] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 22:39:04.485208 21535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:04.485811 22113 ts_tablet_manager.cc:1905] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:04.486675 22115 ts_tablet_manager.cc:1905] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:04.489167 22113 ts_tablet_manager.cc:1918] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:04.489457 22113 log.cc:1196] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/c819e90cf20b476b82ad3217659fbe3d
I20260810 22:39:04.489831 22113 ts_tablet_manager.cc:1939] T c819e90cf20b476b82ad3217659fbe3d P bea2c5f285804f028de79c0868d03696: Deleting consensus metadata
I20260810 22:39:04.490387 22115 ts_tablet_manager.cc:1918] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:04.490553 22115 log.cc:1196] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/c819e90cf20b476b82ad3217659fbe3d
I20260810 22:39:04.490667 21687 tablet_service.cc:1467] Processing CreateTablet for tablet ca306ef67ed244c489ef5e0262193218 (DEFAULT_TABLE table=test-table [id=4bfa1075444748bba98735507432743d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:04.490907 22115 ts_tablet_manager.cc:1939] T c819e90cf20b476b82ad3217659fbe3d P af76116db6354c818e2165d0e6ea3f60: Deleting consensus metadata
I20260810 22:39:04.490938 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca306ef67ed244c489ef5e0262193218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.493083 22091 tablet_bootstrap.cc:492] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: Bootstrap starting.
I20260810 22:39:04.493067 22114 raft_consensus.cc:2258] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:04.493417 21521 catalog_manager.cc:4841] TS af76116db6354c818e2165d0e6ea3f60 (127.19.202.67:46783): tablet c819e90cf20b476b82ad3217659fbe3d (table test-table [id=a06d3ee5174e4541b5e40e2342405b87]) successfully deleted
I20260810 22:39:04.493916 21843 tablet_service.cc:1467] Processing CreateTablet for tablet ca306ef67ed244c489ef5e0262193218 (DEFAULT_TABLE table=test-table [id=4bfa1075444748bba98735507432743d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:04.494164 21843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca306ef67ed244c489ef5e0262193218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.494781 22091 tablet_bootstrap.cc:654] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.494904 22114 ts_tablet_manager.cc:1905] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:04.495911 21520 catalog_manager.cc:4841] TS bea2c5f285804f028de79c0868d03696 (127.19.202.65:43415): tablet c819e90cf20b476b82ad3217659fbe3d (table test-table [id=a06d3ee5174e4541b5e40e2342405b87]) successfully deleted
I20260810 22:39:04.496834 22090 tablet_bootstrap.cc:492] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: Bootstrap starting.
I20260810 22:39:04.497534 22091 tablet_bootstrap.cc:492] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: No bootstrap required, opened a new log
I20260810 22:39:04.497603 22091 ts_tablet_manager.cc:1397] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 22:39:04.498339 22090 tablet_bootstrap.cc:654] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.498447 22114 ts_tablet_manager.cc:1918] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:04.498433 22091 raft_consensus.cc:348] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.498538 22114 log.cc:1196] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/c819e90cf20b476b82ad3217659fbe3d
I20260810 22:39:04.498549 22091 raft_consensus.cc:374] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.498587 22091 raft_consensus.cc:729] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea2c5f285804f028de79c0868d03696, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.498742 22091 consensus_queue.cc:260] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.498852 22114 ts_tablet_manager.cc:1939] T c819e90cf20b476b82ad3217659fbe3d P 603dd7c7197243b2a09bf060901a3587: Deleting consensus metadata
I20260810 22:39:04.499115 22091 ts_tablet_manager.cc:1428] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 22:39:04.500545 22090 tablet_bootstrap.cc:492] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: No bootstrap required, opened a new log
I20260810 22:39:04.500522 21994 tablet_service.cc:1467] Processing CreateTablet for tablet ca306ef67ed244c489ef5e0262193218 (DEFAULT_TABLE table=test-table [id=4bfa1075444748bba98735507432743d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:04.500631 22090 ts_tablet_manager.cc:1397] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 22:39:04.500770 21994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca306ef67ed244c489ef5e0262193218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.501124 22090 raft_consensus.cc:348] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.501183 21521 catalog_manager.cc:4841] TS 603dd7c7197243b2a09bf060901a3587 (127.19.202.66:37693): tablet c819e90cf20b476b82ad3217659fbe3d (table test-table [id=a06d3ee5174e4541b5e40e2342405b87]) successfully deleted
I20260810 22:39:04.501269 22090 raft_consensus.cc:374] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.501333 22090 raft_consensus.cc:729] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603dd7c7197243b2a09bf060901a3587, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.501482 22090 consensus_queue.cc:260] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.501814 22090 ts_tablet_manager.cc:1428] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:39:04.503464 22092 tablet_bootstrap.cc:492] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: Bootstrap starting.
I20260810 22:39:04.504909 22092 tablet_bootstrap.cc:654] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.506319 22092 tablet_bootstrap.cc:492] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: No bootstrap required, opened a new log
I20260810 22:39:04.506423 22092 ts_tablet_manager.cc:1397] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 22:39:04.506873 22092 raft_consensus.cc:348] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.507009 22092 raft_consensus.cc:374] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.507052 22092 raft_consensus.cc:729] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af76116db6354c818e2165d0e6ea3f60, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.507200 22092 consensus_queue.cc:260] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.507463 22092 ts_tablet_manager.cc:1428] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:39:04.782475 22098 raft_consensus.cc:482] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:04.782655 22098 raft_consensus.cc:504] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.783751 22098 leader_election.cc:288] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 603dd7c7197243b2a09bf060901a3587 (127.19.202.66:37693), af76116db6354c818e2165d0e6ea3f60 (127.19.202.67:46783)
I20260810 22:39:04.788051 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.784130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:45613 (local address 127.19.202.66:37693)
0810 22:39:04.784303 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.784308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.784503 (+ 195us) server_negotiation.cc:407] Connection header received
0810 22:39:04.784732 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.784736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.784821 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.784922 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:04.785176 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.785693 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.787071 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.787448 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.787571 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.787645 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.787833 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.787901 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:39:04.788049 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.784556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:37811 (local address 127.19.202.67:46783)
0810 22:39:04.784694 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.784700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.785016 (+ 316us) server_negotiation.cc:407] Connection header received
0810 22:39:04.785147 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.785150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.785198 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.785280 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:04.785550 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.786023 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.787075 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.787455 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.787578 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.787629 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.787850 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.787918 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:39:04.788093 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.784446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:46783 (local address 127.19.202.65:37811)
0810 22:39:04.784924 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.784936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.785060 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.785293 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.785296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.785310 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:04.785417 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.785423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.786153 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.786156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.786960 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.786968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.787574 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.787577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.787665 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.787733 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.787817 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.787901 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":79,"thread_start_us":142,"threads_started":1}
I20260810 22:39:04.788216 22120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.784056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:37693 (local address 127.19.202.65:45613)
0810 22:39:04.784321 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.784338 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.784557 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.784932 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.784935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.784955 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:04.785070 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.785075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.785807 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.785810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.786952 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.786961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.787566 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.787570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.787662 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.787720 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.787806 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.787885 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":29,"thread_start_us":82,"threads_started":1}
I20260810 22:39:04.788707 22016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca306ef67ed244c489ef5e0262193218" candidate_uuid: "bea2c5f285804f028de79c0868d03696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af76116db6354c818e2165d0e6ea3f60" is_pre_election: true
I20260810 22:39:04.788738 21864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca306ef67ed244c489ef5e0262193218" candidate_uuid: "bea2c5f285804f028de79c0868d03696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603dd7c7197243b2a09bf060901a3587" is_pre_election: true
I20260810 22:39:04.788920 21864 raft_consensus.cc:2454] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bea2c5f285804f028de79c0868d03696 in term 0.
I20260810 22:39:04.788964 22016 raft_consensus.cc:2454] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bea2c5f285804f028de79c0868d03696 in term 0.
I20260810 22:39:04.789287 21636 leader_election.cc:302] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603dd7c7197243b2a09bf060901a3587, bea2c5f285804f028de79c0868d03696; no voters:
I20260810 22:39:04.789551 22098 raft_consensus.cc:2781] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:04.789603 22098 raft_consensus.cc:482] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:04.789629 22098 raft_consensus.cc:3037] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:04.790622 22098 raft_consensus.cc:504] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.790958 22098 leader_election.cc:288] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [CANDIDATE]: Term 1 election: Requested vote from peers 603dd7c7197243b2a09bf060901a3587 (127.19.202.66:37693), af76116db6354c818e2165d0e6ea3f60 (127.19.202.67:46783)
I20260810 22:39:04.791342 21864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca306ef67ed244c489ef5e0262193218" candidate_uuid: "bea2c5f285804f028de79c0868d03696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603dd7c7197243b2a09bf060901a3587"
I20260810 22:39:04.791392 22016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca306ef67ed244c489ef5e0262193218" candidate_uuid: "bea2c5f285804f028de79c0868d03696" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af76116db6354c818e2165d0e6ea3f60"
I20260810 22:39:04.791483 21864 raft_consensus.cc:3037] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:04.791536 22016 raft_consensus.cc:3037] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:04.792408 21864 raft_consensus.cc:2454] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bea2c5f285804f028de79c0868d03696 in term 1.
I20260810 22:39:04.792666 22016 raft_consensus.cc:2454] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bea2c5f285804f028de79c0868d03696 in term 1.
I20260810 22:39:04.792692 21636 leader_election.cc:302] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603dd7c7197243b2a09bf060901a3587, bea2c5f285804f028de79c0868d03696; no voters:
I20260810 22:39:04.792922 22098 raft_consensus.cc:2781] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:04.793164 22098 raft_consensus.cc:686] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 1 LEADER]: Becoming Leader. State: Replica: bea2c5f285804f028de79c0868d03696, State: Running, Role: LEADER
I20260810 22:39:04.793504 22098 consensus_queue.cc:237] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.795848 21535 catalog_manager.cc:5495] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 reported cstate change: term changed from 0 to 1, leader changed from <none> to bea2c5f285804f028de79c0868d03696 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "bea2c5f285804f028de79c0868d03696" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } health_report { overall_health: UNKNOWN } } }
Aug 10 22:39:04 dist-test-slave-1x4g krb5kdc[21487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401543, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:04.906840 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.898416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:45525 (local address 127.0.0.1:51112)
0810 22:39:04.898832 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:04.898846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:04.898974 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:04.899230 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:04.899232 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:04.899713 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:04.899804 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.899810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.900567 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.900572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.901150 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:04.901160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.901608 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.901611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:04.901693 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.902264 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:04.902276 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:04.903694 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:04.905218 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:04.905228 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:04.905234 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:04.905442 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:04.905700 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:04.905704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:04.905705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:04.905777 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:04.906130 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:04.906137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:04.906232 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 22:39:04.906339 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 22:39:04.906511 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":215,"threads_started":1}
I20260810 22:39:04.906840 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:04.898457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.19.202.126:45525)
0810 22:39:04.898815 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:04.898893 (+ 78us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:04.898926 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:39:04.899086 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:04.899089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:04.899124 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:04.899210 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:04.899920 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.900464 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.901283 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:04.901502 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:04.901607 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:04.903824 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:04.903827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:04.903829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:04.903847 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:04.905120 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.905534 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.905539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.905541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.905618 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:04.905864 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:04.905867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:04.905868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:04.906033 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:04.906131 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:04.906381 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 22:39:04.906590 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":313,"thread_start_us":228,"threads_started":1}
I20260810 22:39:04.911147 21535 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:51112:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 22:39:04.911821 21535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:04.916577 21843 tablet_service.cc:1467] Processing CreateTablet for tablet 15c47798a8ce481b84a0d2ed2eea71fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa9a19ffa7cf4999a0324312c9599bd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:04.916499 21687 tablet_service.cc:1467] Processing CreateTablet for tablet 15c47798a8ce481b84a0d2ed2eea71fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa9a19ffa7cf4999a0324312c9599bd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:04.916672 21994 tablet_service.cc:1467] Processing CreateTablet for tablet 15c47798a8ce481b84a0d2ed2eea71fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa9a19ffa7cf4999a0324312c9599bd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:04.917009 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15c47798a8ce481b84a0d2ed2eea71fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.917011 21843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15c47798a8ce481b84a0d2ed2eea71fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.917053 21994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15c47798a8ce481b84a0d2ed2eea71fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:04.920073 22092 tablet_bootstrap.cc:492] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: Bootstrap starting.
I20260810 22:39:04.920203 22091 tablet_bootstrap.cc:492] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: Bootstrap starting.
I20260810 22:39:04.920195 22090 tablet_bootstrap.cc:492] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: Bootstrap starting.
I20260810 22:39:04.921216 22092 tablet_bootstrap.cc:654] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.921663 22090 tablet_bootstrap.cc:654] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.922012 22091 tablet_bootstrap.cc:654] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:04.922559 22092 tablet_bootstrap.cc:492] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: No bootstrap required, opened a new log
I20260810 22:39:04.922642 22092 ts_tablet_manager.cc:1397] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 22:39:04.923100 22092 raft_consensus.cc:348] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.923239 22092 raft_consensus.cc:374] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.923273 22092 raft_consensus.cc:729] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af76116db6354c818e2165d0e6ea3f60, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.923345 22090 tablet_bootstrap.cc:492] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: No bootstrap required, opened a new log
I20260810 22:39:04.923437 22090 ts_tablet_manager.cc:1397] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 22:39:04.923398 22092 consensus_queue.cc:260] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.923714 22092 ts_tablet_manager.cc:1428] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:39:04.923779 22101 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:04.923947 22101 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:04.924448 22091 tablet_bootstrap.cc:492] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: No bootstrap required, opened a new log
I20260810 22:39:04.924533 22091 ts_tablet_manager.cc:1397] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 22:39:04.924602 22090 raft_consensus.cc:348] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.924753 22090 raft_consensus.cc:374] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.924862 22090 raft_consensus.cc:729] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603dd7c7197243b2a09bf060901a3587, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.925010 22090 consensus_queue.cc:260] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.925211 22091 raft_consensus.cc:348] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.925318 22091 raft_consensus.cc:374] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:04.925349 22091 raft_consensus.cc:729] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea2c5f285804f028de79c0868d03696, State: Initialized, Role: FOLLOWER
I20260810 22:39:04.925464 22091 consensus_queue.cc:260] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:04.925419 22096 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:04.925588 22096 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:04.925699 22098 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:04.925823 22098 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:04.926508 22090 ts_tablet_manager.cc:1428] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260810 22:39:04.925710 22091 ts_tablet_manager.cc:1428] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:39:05.108058 22096 raft_consensus.cc:482] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:05.108203 22096 raft_consensus.cc:504] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:05.109426 22096 leader_election.cc:288] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bea2c5f285804f028de79c0868d03696 (127.19.202.65:43415), af76116db6354c818e2165d0e6ea3f60 (127.19.202.67:46783)
I20260810 22:39:05.115109 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.109719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:49271 (local address 127.19.202.65:43415)
0810 22:39:05.110002 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.110007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.110056 (+ 49us) server_negotiation.cc:407] Connection header received
0810 22:39:05.110256 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.110259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.110310 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.110404 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:05.110809 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.111801 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.113712 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.114444 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.114553 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.114600 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.114907 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.114978 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":67,"threads_started":1}
I20260810 22:39:05.115106 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.109863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:41549 (local address 127.19.202.67:46783)
0810 22:39:05.110002 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.110007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.110149 (+ 142us) server_negotiation.cc:407] Connection header received
0810 22:39:05.110251 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.110254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.110310 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.110408 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:05.110808 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.111714 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.113712 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.114355 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.114528 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.114587 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.114886 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.114971 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:39:05.115190 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.109697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:46783 (local address 127.19.202.66:41549)
0810 22:39:05.110063 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:05.110077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:05.110156 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:05.110434 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:05.110437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:05.110481 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:05.110689 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.110695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.111827 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.111829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.113579 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.113590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.114521 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.114527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.114721 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.114778 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 22:39:05.114852 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 22:39:05.114963 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":123,"spinlock_wait_cycles":37120,"thread_start_us":72,"threads_started":1}
I20260810 22:39:05.115250 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.109610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:43415 (local address 127.19.202.66:49271)
0810 22:39:05.109968 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:05.109985 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:05.110115 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:05.110434 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:05.110437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:05.110482 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:05.110682 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.110689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.112048 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.112051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.113578 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.113590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.114547 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.114550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.114724 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.114771 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 22:39:05.114852 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 22:39:05.114969 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":89,"spinlock_wait_cycles":5632,"thread_start_us":99,"threads_started":1}
I20260810 22:39:05.115746 22016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15c47798a8ce481b84a0d2ed2eea71fc" candidate_uuid: "603dd7c7197243b2a09bf060901a3587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af76116db6354c818e2165d0e6ea3f60" is_pre_election: true
I20260810 22:39:05.115923 22016 raft_consensus.cc:2454] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 603dd7c7197243b2a09bf060901a3587 in term 0.
I20260810 22:39:05.115978 21709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15c47798a8ce481b84a0d2ed2eea71fc" candidate_uuid: "603dd7c7197243b2a09bf060901a3587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea2c5f285804f028de79c0868d03696" is_pre_election: true
I20260810 22:39:05.116129 21709 raft_consensus.cc:2454] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 603dd7c7197243b2a09bf060901a3587 in term 0.
I20260810 22:39:05.116345 21788 leader_election.cc:302] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603dd7c7197243b2a09bf060901a3587, af76116db6354c818e2165d0e6ea3f60; no voters:
I20260810 22:39:05.116686 22096 raft_consensus.cc:2781] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:05.116742 22096 raft_consensus.cc:482] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:05.116809 22096 raft_consensus.cc:3037] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:05.118162 22096 raft_consensus.cc:504] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:05.118544 22096 leader_election.cc:288] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [CANDIDATE]: Term 1 election: Requested vote from peers bea2c5f285804f028de79c0868d03696 (127.19.202.65:43415), af76116db6354c818e2165d0e6ea3f60 (127.19.202.67:46783)
I20260810 22:39:05.118870 21709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15c47798a8ce481b84a0d2ed2eea71fc" candidate_uuid: "603dd7c7197243b2a09bf060901a3587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea2c5f285804f028de79c0868d03696"
I20260810 22:39:05.118933 22016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15c47798a8ce481b84a0d2ed2eea71fc" candidate_uuid: "603dd7c7197243b2a09bf060901a3587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af76116db6354c818e2165d0e6ea3f60"
I20260810 22:39:05.118988 21709 raft_consensus.cc:3037] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:05.119036 22016 raft_consensus.cc:3037] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:05.120229 21709 raft_consensus.cc:2454] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 603dd7c7197243b2a09bf060901a3587 in term 1.
I20260810 22:39:05.120335 22016 raft_consensus.cc:2454] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 603dd7c7197243b2a09bf060901a3587 in term 1.
I20260810 22:39:05.120589 21787 leader_election.cc:302] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603dd7c7197243b2a09bf060901a3587, bea2c5f285804f028de79c0868d03696; no voters:
I20260810 22:39:05.120788 22096 raft_consensus.cc:2781] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:05.121014 22096 raft_consensus.cc:686] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [term 1 LEADER]: Becoming Leader. State: Replica: 603dd7c7197243b2a09bf060901a3587, State: Running, Role: LEADER
I20260810 22:39:05.121318 22096 consensus_queue.cc:237] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } }
I20260810 22:39:05.122644 22139 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: TxnStatusTablet state changed. Reason: New leader 603dd7c7197243b2a09bf060901a3587. Latest consensus state: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:05.122754 22139 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:05.123214 22141 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 22:39:05.123453 22141 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 22:39:05.124070 21535 catalog_manager.cc:5495] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 reported cstate change: term changed from 0 to 1, leader changed from <none> to 603dd7c7197243b2a09bf060901a3587 (127.19.202.66). New cstate: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:05.160987 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.156428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:37693 (local address 127.0.0.1:53716)
0810 22:39:05.156634 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:05.156652 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:05.156760 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:05.157207 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:05.157209 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:05.157216 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:05.157316 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.157321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.158437 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.158440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.159133 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.159143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.159727 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.159729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.159794 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.159839 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:05.160655 (+ 816us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:05.160669 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 22:39:05.160725 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 22:39:05.160858 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 22:39:05.161008 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.156599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.19.202.66:37693)
0810 22:39:05.156744 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.156749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.156766 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:39:05.156967 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.156971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.157038 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.157158 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:05.157437 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.158275 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.159245 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.159617 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.159722 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.160020 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:05.160543 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:05.160670 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.160782 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.160857 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:39:05.164506 21709 raft_consensus.cc:1261] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Refusing update from remote peer 603dd7c7197243b2a09bf060901a3587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:05.164692 22016 raft_consensus.cc:1261] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Refusing update from remote peer 603dd7c7197243b2a09bf060901a3587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:05.165169 22096 consensus_queue.cc:1035] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:05.165670 22096 consensus_queue.cc:1035] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:05.166836 22098 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: TxnStatusTablet state changed. Reason: New leader 603dd7c7197243b2a09bf060901a3587. Latest consensus state: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:05.166970 22098 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:05.167778 22101 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: TxnStatusTablet state changed. Reason: New leader 603dd7c7197243b2a09bf060901a3587. Latest consensus state: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:05.167891 22101 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:05.169659 22096 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:05.169783 22096 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:05.171345 22098 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:05.171487 22098 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P bea2c5f285804f028de79c0868d03696: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:05.172583 22096 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:05.172690 22096 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P 603dd7c7197243b2a09bf060901a3587: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:05.174250 22101 tablet_replica.cc:439] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "603dd7c7197243b2a09bf060901a3587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea2c5f285804f028de79c0868d03696" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43415 } } peers { permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 } } peers { permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 } } }
I20260810 22:39:05.174378 22101 tablet_replica.cc:442] T 15c47798a8ce481b84a0d2ed2eea71fc P af76116db6354c818e2165d0e6ea3f60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:05.181386 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.176862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.19.202.65:43415)
0810 22:39:05.176996 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.177001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.177229 (+ 228us) server_negotiation.cc:407] Connection header received
0810 22:39:05.177522 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.177525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.177580 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.177673 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:05.177956 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.178491 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.179832 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.180057 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.180145 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.180449 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:05.180980 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:05.181074 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.181163 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.181212 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:39:05.193040 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.189598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:45171 (local address 127.19.202.66:37693)
0810 22:39:05.189767 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.189772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.189965 (+ 193us) server_negotiation.cc:407] Connection header received
0810 22:39:05.190110 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.190114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.190157 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.190236 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:05.190550 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.191003 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.191865 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.192105 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.192213 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.192551 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:05.192625 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:05.192705 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.192860 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.192910 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 22:39:05.193107 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.189395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:37693 (local address 127.19.202.65:45171)
0810 22:39:05.189880 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:05.189893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:05.190010 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:05.190247 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:05.190250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:05.190257 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:05.190447 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.190453 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.191127 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.191130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.191746 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.191755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.192216 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.192220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.192368 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.192411 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:05.192712 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:05.192722 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 22:39:05.192843 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 22:39:05.192929 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":173,"threads_started":1}
I20260810 22:39:05.200300 22016 raft_consensus.cc:1261] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Refusing update from remote peer bea2c5f285804f028de79c0868d03696: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:05.200405 21864 raft_consensus.cc:1261] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Refusing update from remote peer bea2c5f285804f028de79c0868d03696: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:05.200974 22122 consensus_queue.cc:1035] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af76116db6354c818e2165d0e6ea3f60" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:05.201182 22098 consensus_queue.cc:1035] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [LEADER]: Connected to new peer: Peer: permanent_uuid: "603dd7c7197243b2a09bf060901a3587" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:05.226420 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.221649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:42385 (local address 127.19.202.65:43415)
0810 22:39:05.221816 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.221821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.222468 (+ 647us) server_negotiation.cc:407] Connection header received
0810 22:39:05.222578 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.222581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.222638 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.222729 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:05.223114 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.223876 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.225054 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.225276 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.225384 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.225927 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:05.226033 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:05.226111 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.226227 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.226284 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:05.226526 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.221649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:43415 (local address 127.19.202.66:42385)
0810 22:39:05.222347 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:05.222365 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:05.222483 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:05.222763 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:05.222766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:05.222774 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:05.222990 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.222998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.223969 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.223974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.224929 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:05.224939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.225463 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.225467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:05.225687 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.225775 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:05.226133 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:05.226141 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 22:39:05.226203 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 22:39:05.226289 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":411,"threads_started":1}
I20260810 22:39:05.254948 21535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:05.255141 21535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:05.257648 21535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56c8aae4a2824a92864bd01a83a12c6c: Sending DeleteTablet for 3 replicas of tablet ca306ef67ed244c489ef5e0262193218
I20260810 22:39:05.258438 21843 tablet_service.cc:1514] Processing DeleteTablet for tablet ca306ef67ed244c489ef5e0262193218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:05 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:53714
I20260810 22:39:05.258522 21994 tablet_service.cc:1514] Processing DeleteTablet for tablet ca306ef67ed244c489ef5e0262193218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:05 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:40372
I20260810 22:39:05.259276 21687 tablet_service.cc:1514] Processing DeleteTablet for tablet ca306ef67ed244c489ef5e0262193218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:05 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:48928
I20260810 22:39:05.259967 22168 tablet_replica.cc:330] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: stopping tablet replica
I20260810 22:39:05.260174 22168 raft_consensus.cc:2229] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:05.260519 22168 raft_consensus.cc:2258] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:05.260722 22167 tablet_replica.cc:330] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: stopping tablet replica
I20260810 22:39:05.261018 22167 raft_consensus.cc:2229] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:05.261196 22167 raft_consensus.cc:2258] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:05.261318 22168 ts_tablet_manager.cc:1905] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:05.261974 22167 ts_tablet_manager.cc:1905] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:05.264825 22168 ts_tablet_manager.cc:1918] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:05.264967 22168 log.cc:1196] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/ca306ef67ed244c489ef5e0262193218
I20260810 22:39:05.265014 22166 tablet_replica.cc:330] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: stopping tablet replica
I20260810 22:39:05.265360 22168 ts_tablet_manager.cc:1939] T ca306ef67ed244c489ef5e0262193218 P bea2c5f285804f028de79c0868d03696: Deleting consensus metadata
I20260810 22:39:05.265431 22166 raft_consensus.cc:2229] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:05.265697 22166 raft_consensus.cc:2258] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:05.267671 22166 ts_tablet_manager.cc:1905] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:05.267652 22167 ts_tablet_manager.cc:1918] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:05.267755 22167 log.cc:1196] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/ca306ef67ed244c489ef5e0262193218
I20260810 22:39:05.268256 22167 ts_tablet_manager.cc:1939] T ca306ef67ed244c489ef5e0262193218 P af76116db6354c818e2165d0e6ea3f60: Deleting consensus metadata
I20260810 22:39:05.269034 21520 catalog_manager.cc:4841] TS bea2c5f285804f028de79c0868d03696 (127.19.202.65:43415): tablet ca306ef67ed244c489ef5e0262193218 (table test-table [id=4bfa1075444748bba98735507432743d]) successfully deleted
I20260810 22:39:05.269340 21521 catalog_manager.cc:4841] TS af76116db6354c818e2165d0e6ea3f60 (127.19.202.67:46783): tablet ca306ef67ed244c489ef5e0262193218 (table test-table [id=4bfa1075444748bba98735507432743d]) successfully deleted
I20260810 22:39:05.270970 22166 ts_tablet_manager.cc:1918] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:05.271051 22166 log.cc:1196] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/ca306ef67ed244c489ef5e0262193218
I20260810 22:39:05.271307 22166 ts_tablet_manager.cc:1939] T ca306ef67ed244c489ef5e0262193218 P 603dd7c7197243b2a09bf060901a3587: Deleting consensus metadata
I20260810 22:39:05.271950 21521 catalog_manager.cc:4841] TS 603dd7c7197243b2a09bf060901a3587 (127.19.202.66:37693): tablet ca306ef67ed244c489ef5e0262193218 (table test-table [id=4bfa1075444748bba98735507432743d]) successfully deleted
Aug 10 22:39:05 dist-test-slave-1x4g krb5kdc[21487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:05.287593 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.265777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.19.202.65:43415)
0810 22:39:05.265885 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.265889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.273161 (+ 7272us) server_negotiation.cc:407] Connection header received
0810 22:39:05.273256 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.273259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.273395 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.273580 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:05.274498 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.275640 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.277014 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.277361 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.277468 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.281933 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:05.281945 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:05.281966 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:05.281994 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:05.284879 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:05.285458 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:05.285474 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:05.285476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:05.285750 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:05.286127 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:05.286131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:05.286133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:05.286566 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:05.286902 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.287047 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.287215 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 22:39:05.288844 21752 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48958
I20260810 22:39:05.303002 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:05.290817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.19.202.126:45525)
0810 22:39:05.290944 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:05.290949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:05.293078 (+ 2129us) server_negotiation.cc:407] Connection header received
0810 22:39:05.293235 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:05.293238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:05.293301 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:05.293405 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:05.294460 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.295358 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.296406 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:05.296776 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:05.296920 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:05.298918 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:05.298922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:05.298925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:05.298954 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:05.300653 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:05.301267 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:05.301271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:05.301273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:05.301393 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:05.301770 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:05.301773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:05.301775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:05.302034 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:05.302155 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:05.302661 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 22:39:05.302811 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 22:39:05.308550 21535 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51126
I20260810 22:39:05.309868 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 21615
I20260810 22:39:05.322288 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 21769
I20260810 22:39:05.330895 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 21925
I20260810 22:39:05.341063 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 21500
2026-08-10T22:39:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3407 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:05 dist-test-slave-1x4g krb5kdc[22177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:05 dist-test-slave-1x4g krb5kdc[22177](info): set up 2 sockets
Aug 10 22:39:05 dist-test-slave-1x4g krb5kdc[22177](info): commencing operation
krb5kdc: starting...
W20260810 22:39:06.396054 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:06 dist-test-slave-1x4g krb5kdc[22177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:06.513883 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:33883
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:33883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:06.620718 22192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:06.620949 22192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:06.621007 22192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:06.624732 22192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:06.624790 22192 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 22:39:06.624807 22192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:06.624857 22192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:06.624886 22192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:06.624914 22192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:06.629711 22192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:33883
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:33883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:06.630632 22192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:06.631628 22192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:06.639777 22198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:06.639806 22197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:06.639816 22192 server_base.cc:1034] running on GCE node
W20260810 22:39:06.639984 22200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:06.640537 22192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:06.641649 22192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:06.642840 22192 hybrid_clock.cc:648] HybridClock initialized: now 1786401546642821 us; error 39 us; skew 500 ppm
Aug 10 22:39:06 dist-test-slave-1x4g krb5kdc[22177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401546, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:06.645488 22192 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:06.646288 22192 webserver.cc:460] Webserver started at http://127.19.202.126:32941/ using document root <none> and password file <none>
I20260810 22:39:06.646814 22192 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:06.646907 22192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:06.647141 22192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:06.649142 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "ee616e7f86c043958839fea519ec3959"
format_stamp: "Formatted at 2026-08-10 22:39:06 on dist-test-slave-1x4g"
server_key: "efe90742f9e541a5fbf0d06d9ed444a7"
server_key_iv: "9c7fbe8d2bde02dc71972706458291d0"
server_key_version: "encryptionkey@0"
I20260810 22:39:06.649711 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "ee616e7f86c043958839fea519ec3959"
format_stamp: "Formatted at 2026-08-10 22:39:06 on dist-test-slave-1x4g"
server_key: "efe90742f9e541a5fbf0d06d9ed444a7"
server_key_iv: "9c7fbe8d2bde02dc71972706458291d0"
server_key_version: "encryptionkey@0"
I20260810 22:39:06.653168 22192 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:06.655520 22208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:06.656651 22192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:06.656850 22192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "ee616e7f86c043958839fea519ec3959"
format_stamp: "Formatted at 2026-08-10 22:39:06 on dist-test-slave-1x4g"
server_key: "efe90742f9e541a5fbf0d06d9ed444a7"
server_key_iv: "9c7fbe8d2bde02dc71972706458291d0"
server_key_version: "encryptionkey@0"
I20260810 22:39:06.656987 22192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:06.667335 22192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:06.669888 22192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:06.670064 22192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:06.677506 22192 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:33883
I20260810 22:39:06.677528 22275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:33883 every 8 connection(s)
I20260810 22:39:06.678341 22192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:06.679466 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 22192
I20260810 22:39:06.679574 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:06.679852 20265 external_mini_cluster.cc:1428] Setting key c5c32d68d3cf6b8fd1dafa47b4fe6e8d
I20260810 22:39:06.681344 22276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:06.686578 22276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Bootstrap starting.
Aug 10 22:39:06 dist-test-slave-1x4g krb5kdc[22177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:06.689121 22276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:06.690234 22276 log.cc:824] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:06.691978 22276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: No bootstrap required, opened a new log
I20260810 22:39:06.693346 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:06.681161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.19.202.126:33883)
0810 22:39:06.681586 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:06.681600 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:06.681632 (+ 32us) server_negotiation.cc:407] Connection header received
0810 22:39:06.682243 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:06.682258 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:06.682559 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:06.682879 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:06.683669 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.684652 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.685510 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.685954 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.686123 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:06.688847 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:06.688854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:06.688886 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:06.688910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:06.690640 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:06.691102 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:06.691108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:06.691112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:06.691213 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:06.691495 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:06.691499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:06.691500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:06.691851 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:06.692047 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:06.692353 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 22:39:06.692627 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":138,"threads_started":1}
I20260810 22:39:06.695312 22276 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee616e7f86c043958839fea519ec3959" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33883 } }
I20260810 22:39:06.695585 22276 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:06.695665 22276 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee616e7f86c043958839fea519ec3959, State: Initialized, Role: FOLLOWER
I20260810 22:39:06.696030 22276 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee616e7f86c043958839fea519ec3959" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33883 } }
I20260810 22:39:06.696172 22276 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:06.696256 22276 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:06.696369 22276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:06.697297 22276 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee616e7f86c043958839fea519ec3959" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33883 } }
I20260810 22:39:06.697656 22276 leader_election.cc:302] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee616e7f86c043958839fea519ec3959; no voters:
I20260810 22:39:06.698038 22276 leader_election.cc:288] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:06.698112 22282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:06.698351 22282 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [term 1 LEADER]: Becoming Leader. State: Replica: ee616e7f86c043958839fea519ec3959, State: Running, Role: LEADER
I20260810 22:39:06.698643 22282 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee616e7f86c043958839fea519ec3959" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33883 } }
I20260810 22:39:06.699204 22276 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:06.700253 22285 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee616e7f86c043958839fea519ec3959. Latest consensus state: current_term: 1 leader_uuid: "ee616e7f86c043958839fea519ec3959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee616e7f86c043958839fea519ec3959" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33883 } } }
I20260810 22:39:06.700389 22285 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:06.700735 22292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:06.700713 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee616e7f86c043958839fea519ec3959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee616e7f86c043958839fea519ec3959" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33883 } } }
I20260810 22:39:06.700935 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:06.703392 22292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:06.708977 22292 catalog_manager.cc:1347] Generated new cluster ID: 0360a98f73494dc582491e7a024e2569
I20260810 22:39:06.709038 22292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:06.716550 22292 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:06.717454 22292 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:06.726079 22292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Generated new TSK 0
I20260810 22:39:06.726800 22292 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:06.759563 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:33883
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:06.882977 22306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:06.883217 22306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:06.883263 22306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:06.887176 22306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:06.887229 22306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:06.887266 22306 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 22:39:06.887280 22306 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:06.887317 22306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:06.892323 22306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:33883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:06.893356 22306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:06.894327 22306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:06.904068 22314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:06.904044 22311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:06.904036 22312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:06.904279 22306 server_base.cc:1034] running on GCE node
I20260810 22:39:06.904824 22306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:06.905378 22306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:06.906543 22306 hybrid_clock.cc:648] HybridClock initialized: now 1786401546906526 us; error 28 us; skew 500 ppm
Aug 10 22:39:06 dist-test-slave-1x4g krb5kdc[22177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401546, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:06.909374 22306 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:06.910189 22306 webserver.cc:460] Webserver started at http://127.19.202.65:38681/ using document root <none> and password file <none>
I20260810 22:39:06.910728 22306 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:06.910784 22306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:06.910945 22306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:06.912951 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "8a9c66c20fd54e2f93748204320c5e55"
format_stamp: "Formatted at 2026-08-10 22:39:06 on dist-test-slave-1x4g"
server_key: "8affef3694f3881a298c2448e19cf28d"
server_key_iv: "c9635fb33b53cf2204aa3df78f37c61a"
server_key_version: "encryptionkey@0"
I20260810 22:39:06.913486 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "8a9c66c20fd54e2f93748204320c5e55"
format_stamp: "Formatted at 2026-08-10 22:39:06 on dist-test-slave-1x4g"
server_key: "8affef3694f3881a298c2448e19cf28d"
server_key_iv: "c9635fb33b53cf2204aa3df78f37c61a"
server_key_version: "encryptionkey@0"
I20260810 22:39:06.917145 22306 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:06.919422 22322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:06.920531 22306 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:06.920622 22306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "8a9c66c20fd54e2f93748204320c5e55"
format_stamp: "Formatted at 2026-08-10 22:39:06 on dist-test-slave-1x4g"
server_key: "8affef3694f3881a298c2448e19cf28d"
server_key_iv: "c9635fb33b53cf2204aa3df78f37c61a"
server_key_version: "encryptionkey@0"
I20260810 22:39:06.920702 22306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:06.946120 22306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:06.949554 22306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:06.949705 22306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:06.951087 22306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:06.951165 22306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:06.951236 22306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:06.951292 22306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:06.962107 22449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:35757 every 8 connection(s)
Aug 10 22:39:06 dist-test-slave-1x4g krb5kdc[22177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401546, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:06.963213 22306 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:35757
I20260810 22:39:06.964040 22306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:06.966178 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 22306
I20260810 22:39:06.966296 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:06.966647 20265 external_mini_cluster.cc:1428] Setting key a0d5c51cbed9a23003a60e62cbb6d8a7
I20260810 22:39:06.966846 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:06.951930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:33677 (local address 127.19.202.126:33883)
0810 22:39:06.952105 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:06.952112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:06.953471 (+ 1359us) server_negotiation.cc:407] Connection header received
0810 22:39:06.954922 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:06.954926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:06.954991 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:06.955085 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:06.956842 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.957694 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.959591 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.959962 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.960109 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:06.963126 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:06.963133 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:06.963136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:06.963161 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:06.964869 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:06.965341 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:06.965343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:06.965345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:06.965419 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:06.965717 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:06.965719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:06.965720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:06.965902 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:06.966013 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:06.966550 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 22:39:06.966687 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 22:39:06.967723 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:06.952718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33883 (local address 127.19.202.65:33677)
0810 22:39:06.953316 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:06.953376 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:06.954665 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:06.955302 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:06.955312 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:06.955859 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:06.956653 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:06.956677 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.957862 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.957866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:06.959404 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:06.959415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.960162 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.960169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:06.960417 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:06.961084 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:06.961102 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:06.962952 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:06.964979 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:06.964985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:06.965000 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:06.965250 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:06.965502 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:06.965504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:06.965506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:06.965633 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:06.966014 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:06.966018 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:06.966266 (+ 248us) client_negotiation.cc:769] Sending connection context
0810 22:39:06.966450 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 22:39:06.966769 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":139,"threads_started":1}
I20260810 22:39:06.970475 22451 heartbeater.cc:347] Connected to a master server at 127.19.202.126:33883
I20260810 22:39:06.970613 22451 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:06.971133 22451 heartbeater.cc:511] Master 127.19.202.126:33883 requested a full tablet report, sending...
I20260810 22:39:06.972609 22226 ts_manager.cc:194] Registered new tserver with Master: 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757)
I20260810 22:39:06.973897 22226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:33677
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:39:06.980201 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:06.971393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33883 (local address 127.19.202.65:34195)
0810 22:39:06.971703 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:06.971718 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:06.971804 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:06.972342 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:06.972348 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:06.972599 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:06.972762 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:06.972772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.974726 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.974731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:06.975556 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:06.975566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.976200 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.976203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:06.976271 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:06.976880 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:06.976891 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:06.977206 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:06.978844 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:06.978846 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:06.978847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:06.979107 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:06.979364 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:06.979366 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:06.979367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:06.979450 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:06.979749 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:06.979751 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:06.979822 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 22:39:06.979897 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 22:39:06.979994 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":119,"threads_started":1}
I20260810 22:39:06.980273 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:06.971787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:34195 (local address 127.19.202.126:33883)
0810 22:39:06.972003 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:06.972009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:06.972025 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:39:06.972065 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:06.972070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:06.972118 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:06.972213 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:06.972939 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.974606 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.975694 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:06.976069 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:06.976187 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:06.977357 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:06.977361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:06.977364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:06.977395 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:06.978751 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:06.979209 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:06.979212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:06.979213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:06.979284 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:06.979525 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:06.979527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:06.979528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:06.979673 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:06.979743 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:06.979968 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 22:39:06.980098 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":823,"server-negotiator.queue_time_us":128}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:06.995754 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:33883
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:07.104569 22460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:07.104840 22460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:07.104888 22460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:07.108654 22460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:07.108697 22460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:07.108752 22460 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 22:39:07.108767 22460 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:07.108860 22460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:07.113878 22460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:33883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:07.114768 22460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:07.115653 22460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:39:07.125136 22460 server_base.cc:1034] running on GCE node
W20260810 22:39:07.125078 22465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:07.125056 22467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:07.125109 22469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:07.125730 22460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:07.126313 22460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:07.127475 22460 hybrid_clock.cc:648] HybridClock initialized: now 1786401547127463 us; error 23 us; skew 500 ppm
Aug 10 22:39:07 dist-test-slave-1x4g krb5kdc[22177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401547, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:07.130165 22460 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:07.131033 22460 webserver.cc:460] Webserver started at http://127.19.202.66:44659/ using document root <none> and password file <none>
I20260810 22:39:07.131574 22460 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:07.131664 22460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:07.131892 22460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:07.133955 22460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "550fc6001a224a24bd48931d6321ce05"
format_stamp: "Formatted at 2026-08-10 22:39:07 on dist-test-slave-1x4g"
server_key: "b7a0b4efc6b82f95f2a035bae6397dbc"
server_key_iv: "228b871382c5c704b92f5d8306cc53ae"
server_key_version: "encryptionkey@0"
I20260810 22:39:07.134517 22460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "550fc6001a224a24bd48931d6321ce05"
format_stamp: "Formatted at 2026-08-10 22:39:07 on dist-test-slave-1x4g"
server_key: "b7a0b4efc6b82f95f2a035bae6397dbc"
server_key_iv: "228b871382c5c704b92f5d8306cc53ae"
server_key_version: "encryptionkey@0"
I20260810 22:39:07.138180 22460 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:07.140458 22476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:07.141669 22460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:07.141798 22460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "550fc6001a224a24bd48931d6321ce05"
format_stamp: "Formatted at 2026-08-10 22:39:07 on dist-test-slave-1x4g"
server_key: "b7a0b4efc6b82f95f2a035bae6397dbc"
server_key_iv: "228b871382c5c704b92f5d8306cc53ae"
server_key_version: "encryptionkey@0"
I20260810 22:39:07.141904 22460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:07.158910 22460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:07.162016 22460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:07.162242 22460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:07.163642 22460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:07.163712 22460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:07.163781 22460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:07.163830 22460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:07 dist-test-slave-1x4g krb5kdc[22177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401547, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:07.174584 22460 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:38203
I20260810 22:39:07.174646 22603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:38203 every 8 connection(s)
I20260810 22:39:07.175771 22460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:07.178078 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.164454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:37947 (local address 127.19.202.126:33883)
0810 22:39:07.164572 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.164576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.165636 (+ 1060us) server_negotiation.cc:407] Connection header received
0810 22:39:07.166559 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.166565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.166630 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.166729 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:07.167850 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.168771 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.170188 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.170432 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.170550 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.174198 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:07.174201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:07.174204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:07.174235 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:07.176058 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.176683 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.176685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.176687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.176761 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.177158 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.177160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.177162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.177328 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:07.177406 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.177861 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.177955 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:07.178839 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.165009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33883 (local address 127.19.202.66:37947)
0810 22:39:07.165492 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.165546 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.166316 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.166945 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.166955 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.167285 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:07.167723 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.167736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.168934 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.168937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.170060 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.170070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.170598 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.170602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.170902 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.171534 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:07.171556 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:07.174002 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:07.176196 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.176203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.176219 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.176546 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.176900 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.176903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.176905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.177067 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.177425 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:07.177429 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:07.177638 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.177800 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.178038 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":140,"threads_started":1}
I20260810 22:39:07.179908 22604 heartbeater.cc:347] Connected to a master server at 127.19.202.126:33883
I20260810 22:39:07.180032 22604 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:07.180531 22604 heartbeater.cc:511] Master 127.19.202.126:33883 requested a full tablet report, sending...
I20260810 22:39:07.181583 22227 ts_manager.cc:194] Registered new tserver with Master: 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203)
I20260810 22:39:07.181855 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 22460
I20260810 22:39:07.181980 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:07.182291 20265 external_mini_cluster.cc:1428] Setting key 9d8a9ec5ec9205bfd88a1f90cc135796
I20260810 22:39:07.182379 22227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:37947
I20260810 22:39:07.189976 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.182108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33883 (local address 127.19.202.66:50403)
0810 22:39:07.182783 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.182797 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.182891 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.183137 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.183140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.183375 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:07.183497 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.183503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.184377 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.184379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.184976 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.184984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.185586 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.185592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.185701 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.186191 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:07.186208 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:07.186603 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:07.188400 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.188402 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.188404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.188580 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.188936 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.188939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.188940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.189053 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.189567 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:07.189569 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:07.189640 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.189724 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.189815 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":92,"threads_started":1}
I20260810 22:39:07.190140 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.182180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:50403 (local address 127.19.202.126:33883)
0810 22:39:07.182288 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.182291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.182864 (+ 573us) server_negotiation.cc:407] Connection header received
0810 22:39:07.182977 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.182979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.183023 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.183102 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:07.183603 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.184238 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.185084 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.185459 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.185572 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.186730 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:07.186732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:07.186734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:07.186753 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:07.188270 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.188679 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.188685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.188690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.188840 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.189159 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.189165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.189169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.189469 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:07.189569 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.189811 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.189930 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:07.209514 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:33883
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:07.319170 22615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:07.319422 22615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:07.319497 22615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:07.323289 22615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:07.323374 22615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:07.323449 22615 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 22:39:07.323493 22615 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:07.323562 22615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:07.328441 22615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:33883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:07.329442 22615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:07.330394 22615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:07.339659 22621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:07.339685 22623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:07.339709 22620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:07.339742 22615 server_base.cc:1034] running on GCE node
I20260810 22:39:07.340214 22615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:07.340885 22615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:07.342077 22615 hybrid_clock.cc:648] HybridClock initialized: now 1786401547342065 us; error 33 us; skew 500 ppm
Aug 10 22:39:07 dist-test-slave-1x4g krb5kdc[22177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401547, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:07.344815 22615 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:07.345731 22615 webserver.cc:460] Webserver started at http://127.19.202.67:34395/ using document root <none> and password file <none>
I20260810 22:39:07.346237 22615 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:07.346308 22615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:07.346469 22615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:07.348234 22615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "2a65dd6d024f46f1b7733d6b933608fa"
format_stamp: "Formatted at 2026-08-10 22:39:07 on dist-test-slave-1x4g"
server_key: "b21ca14a4e471fcbd087fd135a248c76"
server_key_iv: "2c28c8d47df79092cda56e8949760405"
server_key_version: "encryptionkey@0"
I20260810 22:39:07.348760 22615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "2a65dd6d024f46f1b7733d6b933608fa"
format_stamp: "Formatted at 2026-08-10 22:39:07 on dist-test-slave-1x4g"
server_key: "b21ca14a4e471fcbd087fd135a248c76"
server_key_iv: "2c28c8d47df79092cda56e8949760405"
server_key_version: "encryptionkey@0"
I20260810 22:39:07.352313 22615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:07.354614 22631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:07.356242 22615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:07.356372 22615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "2a65dd6d024f46f1b7733d6b933608fa"
format_stamp: "Formatted at 2026-08-10 22:39:07 on dist-test-slave-1x4g"
server_key: "b21ca14a4e471fcbd087fd135a248c76"
server_key_iv: "2c28c8d47df79092cda56e8949760405"
server_key_version: "encryptionkey@0"
I20260810 22:39:07.356494 22615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:07.368391 22615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:07.371147 22615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:07.371284 22615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:07.372547 22615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:07.372593 22615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:07.372630 22615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:07.372689 22615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:07 dist-test-slave-1x4g krb5kdc[22177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401547, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:07.383446 22615 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:45507
I20260810 22:39:07.383500 22758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:45507 every 8 connection(s)
I20260810 22:39:07.384348 22615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:07.385442 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 22615
I20260810 22:39:07.385571 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:07.385841 20265 external_mini_cluster.cc:1428] Setting key 98368b60646d35e1faadd739700ea65c
I20260810 22:39:07.386444 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.373432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:49255 (local address 127.19.202.126:33883)
0810 22:39:07.373577 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.373581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.374329 (+ 748us) server_negotiation.cc:407] Connection header received
0810 22:39:07.375260 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.375263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.375322 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.375425 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:07.377205 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.377819 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.379105 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.379327 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.379436 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.382736 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:07.382739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:07.382740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:07.382764 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:07.384240 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.384840 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.384843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.384845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.384930 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.385338 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.385341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.385343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.385529 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:07.385642 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.386159 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.386279 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:39:07.387131 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.373776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33883 (local address 127.19.202.67:49255)
0810 22:39:07.374181 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.374226 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.375010 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.375776 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.375788 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.376279 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:07.377016 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.377041 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.377942 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.377945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.378988 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.378998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.379485 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.379489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.379655 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.380216 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:07.380238 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:07.382575 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:07.384346 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.384352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.384364 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.384686 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.385063 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.385067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.385068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.385240 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.385642 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:07.385648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:07.385869 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.386113 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.386404 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":123,"threads_started":1}
I20260810 22:39:07.388437 22760 heartbeater.cc:347] Connected to a master server at 127.19.202.126:33883
I20260810 22:39:07.388576 22760 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:07.389163 22760 heartbeater.cc:511] Master 127.19.202.126:33883 requested a full tablet report, sending...
I20260810 22:39:07.390168 22224 ts_manager.cc:194] Registered new tserver with Master: 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:07.390887 22224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:49255
I20260810 22:39:07.397205 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.390582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33883 (local address 127.19.202.67:49049)
0810 22:39:07.390954 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.390980 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.391101 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.391364 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.391367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.391563 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:07.391661 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.391666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.392629 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.392632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.393247 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.393255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.393682 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.393685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.393768 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.394273 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:07.394285 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:07.394572 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:07.395961 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.395964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.395965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.396142 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.396387 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.396389 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.396390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.396459 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.396831 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:07.396834 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:07.396909 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.396991 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.397087 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":102,"threads_started":1}
I20260810 22:39:07.397229 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.390679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:49049 (local address 127.19.202.126:33883)
0810 22:39:07.390789 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.390793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.391041 (+ 248us) server_negotiation.cc:407] Connection header received
0810 22:39:07.391205 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.391208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.391254 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.391328 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:07.391770 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.392507 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.393347 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.393557 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.393636 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.394681 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:07.394683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:07.394685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:07.394700 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:07.395869 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.396223 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.396225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.396226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.396294 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.396529 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.396531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.396532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.396698 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:07.396769 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.397051 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.397135 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:39:07.400838 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:39:07 dist-test-slave-1x4g krb5kdc[22177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:39:07 dist-test-slave-1x4g krb5kdc[22177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:07.427357 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.419816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.19.202.126:33883)
0810 22:39:07.419940 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.419943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.419954 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:39:07.419986 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.419987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.420019 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.420103 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:07.420783 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.421309 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.422080 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.422311 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.422427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.424819 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:07.424826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:07.424830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:07.424863 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:07.426036 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.426402 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.426405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.426406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.426481 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.426698 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.426700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.426701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.426855 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:07.426943 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.427121 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.427238 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 22:39:07.429966 22224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:07.432950 22224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:07.446832 22777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.440980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.0.0.1:34250)
0810 22:39:07.441300 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.441347 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.441464 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.442547 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.442553 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.442574 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:07.442707 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.442714 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.444323 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.444326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.445411 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.445420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.446299 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.446302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.446412 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.446454 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.446575 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.446679 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":61,"thread_start_us":75,"threads_started":1}
I20260810 22:39:07.447101 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.441335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:38203 (local address 127.0.0.1:54602)
0810 22:39:07.441693 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.441705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.441776 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.442748 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.442753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.442776 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:07.442869 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.442875 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.444175 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.444178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.445453 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.445465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.446576 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.446580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.446659 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.446678 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.446748 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.446799 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":73,"thread_start_us":98,"threads_started":1}
I20260810 22:39:07.447244 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.441739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54602 (local address 127.19.202.66:38203)
0810 22:39:07.442234 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.442252 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.442275 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:39:07.442367 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.442387 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.442579 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.442767 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:07.443017 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.444045 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.445599 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.446385 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.446547 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.446676 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.446774 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.446867 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":90,"threads_started":1}
I20260810 22:39:07.447467 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.441659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45507 (local address 127.0.0.1:33836)
0810 22:39:07.442203 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.442216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.442290 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.442936 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.442939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.442957 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:07.443066 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.443078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.444002 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.444006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.445462 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.445468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.446743 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.446747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.446842 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.446864 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.447042 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.447259 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":89,"thread_start_us":79,"threads_started":1}
I20260810 22:39:07.447821 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.442073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.19.202.67:45507)
0810 22:39:07.442512 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.442523 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.442543 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:39:07.442613 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.442618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.442782 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.442934 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:07.443186 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.443898 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.445599 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.446392 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.446508 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.446676 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.447625 (+ 949us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.447705 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":137,"threads_started":1}
I20260810 22:39:07.447988 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.441105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.19.202.65:35757)
0810 22:39:07.442013 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.442031 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.442055 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:39:07.442135 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.442140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.442387 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.442559 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:07.442838 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.443990 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.445521 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.446114 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.446248 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.446429 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.447021 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.447117 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":87,"threads_started":1}
I20260810 22:39:07.450372 22687 tablet_service.cc:1467] Processing CreateTablet for tablet 8d309076791e4619a1b6fc41ce56fe67 (DEFAULT_TABLE table=test-table [id=0d002353270841168b537d4b18e51982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:07.450445 22523 tablet_service.cc:1467] Processing CreateTablet for tablet 8d309076791e4619a1b6fc41ce56fe67 (DEFAULT_TABLE table=test-table [id=0d002353270841168b537d4b18e51982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:07.450445 22376 tablet_service.cc:1467] Processing CreateTablet for tablet 8d309076791e4619a1b6fc41ce56fe67 (DEFAULT_TABLE table=test-table [id=0d002353270841168b537d4b18e51982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:07.451347 22687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d309076791e4619a1b6fc41ce56fe67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.451401 22376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d309076791e4619a1b6fc41ce56fe67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.451399 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d309076791e4619a1b6fc41ce56fe67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.458057 22784 tablet_bootstrap.cc:492] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Bootstrap starting.
I20260810 22:39:07.459338 22786 tablet_bootstrap.cc:492] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Bootstrap starting.
I20260810 22:39:07.459867 22785 tablet_bootstrap.cc:492] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Bootstrap starting.
I20260810 22:39:07.460285 22784 tablet_bootstrap.cc:654] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.461391 22786 tablet_bootstrap.cc:654] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.461829 22784 log.cc:824] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:07.461936 22785 tablet_bootstrap.cc:654] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.462218 22786 log.cc:824] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:07.462790 22785 log.cc:824] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:07.464107 22784 tablet_bootstrap.cc:492] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: No bootstrap required, opened a new log
I20260810 22:39:07.464296 22784 ts_tablet_manager.cc:1397] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 22:39:07.464505 22786 tablet_bootstrap.cc:492] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: No bootstrap required, opened a new log
I20260810 22:39:07.464695 22786 ts_tablet_manager.cc:1397] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 22:39:07.464929 22785 tablet_bootstrap.cc:492] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: No bootstrap required, opened a new log
I20260810 22:39:07.465102 22785 ts_tablet_manager.cc:1397] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:39:07.467600 22786 raft_consensus.cc:348] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.467828 22786 raft_consensus.cc:374] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.467880 22786 raft_consensus.cc:729] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550fc6001a224a24bd48931d6321ce05, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.468163 22784 raft_consensus.cc:348] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.468312 22786 consensus_queue.cc:260] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.468425 22784 raft_consensus.cc:374] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.468487 22784 raft_consensus.cc:729] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a65dd6d024f46f1b7733d6b933608fa, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.468932 22604 heartbeater.cc:503] Master 127.19.202.126:33883 was elected leader, sending a full tablet report...
I20260810 22:39:07.469019 22785 raft_consensus.cc:348] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.469242 22786 ts_tablet_manager.cc:1428] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260810 22:39:07.469280 22785 raft_consensus.cc:374] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.469327 22785 raft_consensus.cc:729] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9c66c20fd54e2f93748204320c5e55, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.469597 22784 consensus_queue.cc:260] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.469732 22785 consensus_queue.cc:260] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.470482 22784 ts_tablet_manager.cc:1428] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:39:07.470579 22785 ts_tablet_manager.cc:1428] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:39:07.470781 22760 heartbeater.cc:503] Master 127.19.202.126:33883 was elected leader, sending a full tablet report...
I20260810 22:39:07.470881 22451 heartbeater.cc:503] Master 127.19.202.126:33883 was elected leader, sending a full tablet report...
W20260810 22:39:07.635504 22761 tablet.cc:2369] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:39:07.677565 22605 tablet.cc:2369] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:39:07.678996 22790 raft_consensus.cc:482] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:07.679188 22790 raft_consensus.cc:504] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.680478 22790 leader_election.cc:288] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:07.684957 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.680806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:36187 (local address 127.19.202.65:35757)
0810 22:39:07.680930 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.680935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.681089 (+ 154us) server_negotiation.cc:407] Connection header received
0810 22:39:07.681472 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.681488 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.681563 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.681647 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:07.681882 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.682439 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.683868 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.684398 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.684508 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.684557 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.684761 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.684858 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:07.685101 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.681215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:52721 (local address 127.19.202.67:45507)
0810 22:39:07.681348 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.681352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.681531 (+ 179us) server_negotiation.cc:407] Connection header received
0810 22:39:07.681647 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.681650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.681692 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.681759 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:07.682027 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.682811 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.684078 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.684444 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.684562 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.684615 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.684901 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.684975 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:07.684983 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.680674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.19.202.66:36187)
0810 22:39:07.681007 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.681022 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.681175 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.681658 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.681660 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.681671 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:07.681780 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.681785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.682551 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.682556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.683752 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.683762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.684518 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.684524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.684616 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.684667 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.684744 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.684830 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":4,"thread_start_us":120,"threads_started":1}
I20260810 22:39:07.685434 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.681100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45507 (local address 127.19.202.66:52721)
0810 22:39:07.681411 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.681439 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.681554 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.681784 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.681787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.681803 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:07.681917 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.681923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.682933 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.682936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.683982 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.683989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.684628 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.684631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.684754 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.684800 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.685220 (+ 420us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.685288 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":47,"thread_start_us":54,"threads_started":1}
I20260810 22:39:07.685959 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d309076791e4619a1b6fc41ce56fe67" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa" is_pre_election: true
I20260810 22:39:07.686219 22708 raft_consensus.cc:2454] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 0.
I20260810 22:39:07.686635 22480 leader_election.cc:302] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a65dd6d024f46f1b7733d6b933608fa, 550fc6001a224a24bd48931d6321ce05; no voters:
I20260810 22:39:07.686650 22397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d309076791e4619a1b6fc41ce56fe67" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9c66c20fd54e2f93748204320c5e55" is_pre_election: true
I20260810 22:39:07.686872 22397 raft_consensus.cc:2454] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 0.
I20260810 22:39:07.686887 22790 raft_consensus.cc:2781] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:07.686959 22790 raft_consensus.cc:482] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:07.687009 22790 raft_consensus.cc:3037] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:07.688505 22790 raft_consensus.cc:504] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.688860 22790 leader_election.cc:288] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 election: Requested vote from peers 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:07.689172 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d309076791e4619a1b6fc41ce56fe67" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa"
I20260810 22:39:07.689150 22397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d309076791e4619a1b6fc41ce56fe67" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9c66c20fd54e2f93748204320c5e55"
I20260810 22:39:07.689271 22708 raft_consensus.cc:3037] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:07.689294 22397 raft_consensus.cc:3037] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:07.690215 22397 raft_consensus.cc:2454] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 1.
I20260810 22:39:07.690514 22480 leader_election.cc:302] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550fc6001a224a24bd48931d6321ce05, 8a9c66c20fd54e2f93748204320c5e55; no voters:
I20260810 22:39:07.690703 22790 raft_consensus.cc:2781] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:07.690902 22790 raft_consensus.cc:686] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 550fc6001a224a24bd48931d6321ce05, State: Running, Role: LEADER
I20260810 22:39:07.691047 22708 raft_consensus.cc:2454] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 1.
I20260810 22:39:07.691218 22790 consensus_queue.cc:237] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } }
I20260810 22:39:07.694423 22227 catalog_manager.cc:5495] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 550fc6001a224a24bd48931d6321ce05 (127.19.202.66). New cstate: current_term: 1 leader_uuid: "550fc6001a224a24bd48931d6321ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } health_report { overall_health: HEALTHY } } }
W20260810 22:39:07.715547 22452 tablet.cc:2369] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 22:39:07 dist-test-slave-1x4g krb5kdc[22177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401546, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:07.758774 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.750425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33883 (local address 127.0.0.1:34420)
0810 22:39:07.750741 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.750754 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.750850 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.751078 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.751080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.751488 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:07.751599 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.751604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.752306 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.752308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.752943 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.752952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.753358 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.753360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.753430 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.754064 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:07.754076 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:07.755481 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:07.757155 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.757166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.757173 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.757400 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.757689 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:07.757692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:07.757693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:07.757772 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:07.758145 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:07.758153 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:07.758265 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.758379 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.758552 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":90,"threads_started":1}
I20260810 22:39:07.758773 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.750495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34420 (local address 127.19.202.126:33883)
0810 22:39:07.750612 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.750616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.750824 (+ 208us) server_negotiation.cc:407] Connection header received
0810 22:39:07.750935 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.750937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.750976 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.751041 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:07.751695 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.752212 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.753043 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.753251 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.753340 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.755613 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:07.755616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:07.755618 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:07.755636 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:07.757049 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.757508 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.757513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.757515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.757591 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:07.757859 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:07.757864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:07.757866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:07.758040 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:07.758145 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.758436 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.758624 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":35}
I20260810 22:39:07.762895 22227 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:34420:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 22:39:07.763612 22227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:07.768322 22523 tablet_service.cc:1467] Processing CreateTablet for tablet 183ac76d1dfc4f40b2302bd01e48a6a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4bbf92c28f4a4a39b773ecd150aa130c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:07.768322 22376 tablet_service.cc:1467] Processing CreateTablet for tablet 183ac76d1dfc4f40b2302bd01e48a6a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4bbf92c28f4a4a39b773ecd150aa130c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:07.768611 22687 tablet_service.cc:1467] Processing CreateTablet for tablet 183ac76d1dfc4f40b2302bd01e48a6a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4bbf92c28f4a4a39b773ecd150aa130c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:07.769090 22376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183ac76d1dfc4f40b2302bd01e48a6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.769091 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183ac76d1dfc4f40b2302bd01e48a6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.769330 22687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183ac76d1dfc4f40b2302bd01e48a6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.772068 22784 tablet_bootstrap.cc:492] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: Bootstrap starting.
I20260810 22:39:07.772076 22786 tablet_bootstrap.cc:492] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: Bootstrap starting.
I20260810 22:39:07.772264 22785 tablet_bootstrap.cc:492] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: Bootstrap starting.
I20260810 22:39:07.773065 22784 tablet_bootstrap.cc:654] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.773131 22786 tablet_bootstrap.cc:654] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.773213 22785 tablet_bootstrap.cc:654] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.774570 22784 tablet_bootstrap.cc:492] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: No bootstrap required, opened a new log
I20260810 22:39:07.774685 22784 ts_tablet_manager.cc:1397] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:07.774925 22785 tablet_bootstrap.cc:492] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: No bootstrap required, opened a new log
I20260810 22:39:07.775012 22785 ts_tablet_manager.cc:1397] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:07.775161 22786 tablet_bootstrap.cc:492] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: No bootstrap required, opened a new log
I20260810 22:39:07.775238 22786 ts_tablet_manager.cc:1397] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 22:39:07.775177 22784 raft_consensus.cc:348] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.775281 22784 raft_consensus.cc:374] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.775316 22784 raft_consensus.cc:729] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a65dd6d024f46f1b7733d6b933608fa, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.775465 22784 consensus_queue.cc:260] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.775502 22785 raft_consensus.cc:348] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.775611 22785 raft_consensus.cc:374] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.775655 22785 raft_consensus.cc:729] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9c66c20fd54e2f93748204320c5e55, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.775688 22786 raft_consensus.cc:348] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.775810 22786 raft_consensus.cc:374] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.775779 22785 consensus_queue.cc:260] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.775765 22791 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.775853 22786 raft_consensus.cc:729] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550fc6001a224a24bd48931d6321ce05, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.775899 22791 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:07.775991 22786 consensus_queue.cc:260] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.776047 22785 ts_tablet_manager.cc:1428] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:07.776242 22792 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.776412 22792 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:07.776562 22784 ts_tablet_manager.cc:1428] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:07.776693 22790 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.776849 22790 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:07.777458 22786 ts_tablet_manager.cc:1428] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 22:39:07.826236 22792 raft_consensus.cc:482] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:07.826357 22792 raft_consensus.cc:504] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.827668 22792 leader_election.cc:288] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:07.833503 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.827875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:38203 (local address 127.19.202.65:55527)
0810 22:39:07.828247 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.828261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.828359 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.829057 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.829060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.829106 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:07.829317 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.829327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.830374 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.830382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.831920 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.831931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.833009 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.833014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.833140 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.833168 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.833247 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.833317 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260810 22:39:07.833544 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.828244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:55527 (local address 127.19.202.66:38203)
0810 22:39:07.828704 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.828710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.828747 (+ 37us) server_negotiation.cc:407] Connection header received
0810 22:39:07.828846 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.828849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.828920 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.828996 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:07.829492 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.830219 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.832065 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.832836 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.832969 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.833044 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.833290 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.833399 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339}
I20260810 22:39:07.834144 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.828628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:59433 (local address 127.19.202.67:45507)
0810 22:39:07.828868 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.828872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.828908 (+ 36us) server_negotiation.cc:407] Connection header received
0810 22:39:07.828957 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.828963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.829006 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.829065 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:07.829999 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.830802 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.832710 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.833383 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.833491 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.833553 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.833885 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.833985 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 22:39:07.834405 22536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183ac76d1dfc4f40b2302bd01e48a6a4" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550fc6001a224a24bd48931d6321ce05" is_pre_election: true
I20260810 22:39:07.834555 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.828285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45507 (local address 127.19.202.65:59433)
0810 22:39:07.828640 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.828654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.828745 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.829140 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.829144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.829174 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:07.829811 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.829826 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.831189 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.831194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.832584 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.832594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.833618 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.833621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.833735 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.833764 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.834344 (+ 580us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.834426 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260810 22:39:07.834669 22536 raft_consensus.cc:2454] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 0.
I20260810 22:39:07.834990 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183ac76d1dfc4f40b2302bd01e48a6a4" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa" is_pre_election: true
I20260810 22:39:07.835150 22708 raft_consensus.cc:2454] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 0.
I20260810 22:39:07.835557 22326 leader_election.cc:302] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a65dd6d024f46f1b7733d6b933608fa, 8a9c66c20fd54e2f93748204320c5e55; no voters:
I20260810 22:39:07.835860 22792 raft_consensus.cc:2781] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:07.835938 22792 raft_consensus.cc:482] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:07.835994 22792 raft_consensus.cc:3037] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:07.837756 22792 raft_consensus.cc:504] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.838164 22792 leader_election.cc:288] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 election: Requested vote from peers 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:07.838433 22536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183ac76d1dfc4f40b2302bd01e48a6a4" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550fc6001a224a24bd48931d6321ce05"
I20260810 22:39:07.838490 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183ac76d1dfc4f40b2302bd01e48a6a4" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa"
I20260810 22:39:07.838551 22536 raft_consensus.cc:3037] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:07.838593 22708 raft_consensus.cc:3037] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:07.839906 22536 raft_consensus.cc:2454] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 1.
I20260810 22:39:07.840194 22324 leader_election.cc:302] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550fc6001a224a24bd48931d6321ce05, 8a9c66c20fd54e2f93748204320c5e55; no voters:
I20260810 22:39:07.840378 22792 raft_consensus.cc:2781] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:07.840582 22792 raft_consensus.cc:686] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 LEADER]: Becoming Leader. State: Replica: 8a9c66c20fd54e2f93748204320c5e55, State: Running, Role: LEADER
I20260810 22:39:07.840981 22792 consensus_queue.cc:237] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.842466 22708 raft_consensus.cc:2454] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 1.
I20260810 22:39:07.843657 22810 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: TxnStatusTablet state changed. Reason: New leader 8a9c66c20fd54e2f93748204320c5e55. Latest consensus state: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.843778 22810 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:07.844053 22812 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 22:39:07.844259 22812 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 22:39:07.844929 22226 catalog_manager.cc:5495] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:07.876276 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.872484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.19.202.65:35757)
0810 22:39:07.872608 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.872611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.872696 (+ 85us) server_negotiation.cc:407] Connection header received
0810 22:39:07.872944 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.872947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.873003 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.873087 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:07.873382 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.873857 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.874983 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.875174 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.875244 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.875539 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:07.875949 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:07.876027 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.876126 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.876170 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:07.876331 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.872364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.0.0.1:34266)
0810 22:39:07.872621 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.872635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.872762 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.873129 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.873132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.873140 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:07.873261 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.873267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.873975 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.873978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.874875 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.874884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.875266 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.875270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.875355 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.875408 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:07.876033 (+ 625us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:07.876048 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.876119 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.876199 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260810 22:39:07.881686 22536 raft_consensus.cc:1261] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Refusing update from remote peer 8a9c66c20fd54e2f93748204320c5e55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:07.881690 22708 raft_consensus.cc:1261] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Refusing update from remote peer 8a9c66c20fd54e2f93748204320c5e55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:07.882378 22810 consensus_queue.cc:1035] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:07.882567 22792 consensus_queue.cc:1035] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:07.885625 22790 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: TxnStatusTablet state changed. Reason: New leader 8a9c66c20fd54e2f93748204320c5e55. Latest consensus state: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.885751 22790 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:07.887316 22791 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: TxnStatusTablet state changed. Reason: New leader 8a9c66c20fd54e2f93748204320c5e55. Latest consensus state: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.887452 22791 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:07.889261 22810 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.889364 22810 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:07.889492 22810 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.889557 22810 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 8a9c66c20fd54e2f93748204320c5e55: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:07.890592 22790 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.890691 22790 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 550fc6001a224a24bd48931d6321ce05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:07.893469 22817 mvcc.cc:205] Tried to move back new op lower bound from 7317100740115853312 to 7317100739959492608. Current Snapshot: MvccSnapshot[applied={T|T < 7317100740115853312}]
I20260810 22:39:07.894902 22791 tablet_replica.cc:439] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } } }
I20260810 22:39:07.895005 22791 tablet_replica.cc:442] T 183ac76d1dfc4f40b2302bd01e48a6a4 P 2a65dd6d024f46f1b7733d6b933608fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:07.901158 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.897206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.19.202.66:38203)
0810 22:39:07.897346 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.897350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.897475 (+ 125us) server_negotiation.cc:407] Connection header received
0810 22:39:07.897649 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.897651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.897698 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.897785 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:07.898041 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.898478 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.899666 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.899974 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.900053 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.900355 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:07.900727 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:07.900858 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.900978 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.901030 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:39:07.911381 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.907936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:33581 (local address 127.19.202.65:35757)
0810 22:39:07.908058 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.908062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.908215 (+ 153us) server_negotiation.cc:407] Connection header received
0810 22:39:07.908310 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.908312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.908350 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.908430 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:07.908673 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.909244 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.910418 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.910615 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.910687 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.911022 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:07.911081 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:07.911127 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.911239 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.911281 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:07.911459 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.907855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.19.202.66:33581)
0810 22:39:07.908135 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.908148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.908241 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.908458 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.908461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.908468 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:07.908589 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.908595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.909345 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.909348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.910304 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.910314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.910714 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.910717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.910816 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.910872 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:07.911152 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:07.911158 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.911222 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.911299 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
I20260810 22:39:07.918175 22708 raft_consensus.cc:1261] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Refusing update from remote peer 550fc6001a224a24bd48931d6321ce05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:07.918366 22397 raft_consensus.cc:1261] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Refusing update from remote peer 550fc6001a224a24bd48931d6321ce05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:07.918701 22797 consensus_queue.cc:1035] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:07.918885 22790 consensus_queue.cc:1035] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:07.922325 22816 mvcc.cc:205] Tried to move back new op lower bound from 7317100740268253184 to 7317100739345715200. Current Snapshot: MvccSnapshot[applied={T|T < 7317100740268253184}]
I20260810 22:39:07.940557 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.937162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:38763 (local address 127.19.202.66:38203)
0810 22:39:07.937279 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:07.937282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:07.937517 (+ 235us) server_negotiation.cc:407] Connection header received
0810 22:39:07.937625 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:07.937627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:07.937663 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:07.937723 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:07.937997 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.938534 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.939551 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.939774 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.939881 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.940151 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:07.940228 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:07.940304 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:07.940391 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 22:39:07.940437 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:39:07.940603 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:07.937088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:38203 (local address 127.19.202.65:38763)
0810 22:39:07.937432 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:07.937447 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:07.937551 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:07.937751 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:07.937768 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:07.937775 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:07.937884 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.937890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.938653 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.938656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.939428 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:07.939438 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:07.939886 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:07.939890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:07.939975 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:07.940024 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:07.940312 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:07.940321 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 22:39:07.940381 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 22:39:07.940465 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":91,"threads_started":1}
I20260810 22:39:07.968499 22226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:07.968693 22226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:07.971495 22226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Sending DeleteTablet for 3 replicas of tablet 8d309076791e4619a1b6fc41ce56fe67
I20260810 22:39:07.972332 22376 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d309076791e4619a1b6fc41ce56fe67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:07 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:34250
I20260810 22:39:07.973008 22523 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d309076791e4619a1b6fc41ce56fe67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:07 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:54602
I20260810 22:39:07.973285 22687 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d309076791e4619a1b6fc41ce56fe67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:07 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:33836
I20260810 22:39:07.973315 22226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:07.973732 22226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:07.978448 22523 tablet_service.cc:1467] Processing CreateTablet for tablet 72d7417203714e3da0733fd6b32fb2ce (DEFAULT_TABLE table=test-table [id=29d05cb92a214c2daefcb3e3e301c173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:07.978734 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d7417203714e3da0733fd6b32fb2ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.978876 22843 tablet_replica.cc:330] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: stopping tablet replica
I20260810 22:39:07.979219 22843 raft_consensus.cc:2229] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:07.979684 22843 raft_consensus.cc:2258] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:07.979965 22841 tablet_replica.cc:330] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: stopping tablet replica
I20260810 22:39:07.981218 22786 tablet_bootstrap.cc:492] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: Bootstrap starting.
I20260810 22:39:07.981356 22841 raft_consensus.cc:2229] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:07.981575 22841 raft_consensus.cc:2258] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:07.982131 22786 tablet_bootstrap.cc:654] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.983125 22841 ts_tablet_manager.cc:1905] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:07.983968 22687 tablet_service.cc:1467] Processing CreateTablet for tablet 72d7417203714e3da0733fd6b32fb2ce (DEFAULT_TABLE table=test-table [id=29d05cb92a214c2daefcb3e3e301c173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:07.984164 22843 ts_tablet_manager.cc:1905] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:07.984273 22687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d7417203714e3da0733fd6b32fb2ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.985481 22786 tablet_bootstrap.cc:492] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: No bootstrap required, opened a new log
I20260810 22:39:07.985564 22786 ts_tablet_manager.cc:1397] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 22:39:07.983348 22376 tablet_service.cc:1467] Processing CreateTablet for tablet 72d7417203714e3da0733fd6b32fb2ce (DEFAULT_TABLE table=test-table [id=29d05cb92a214c2daefcb3e3e301c173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:07.985775 22376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d7417203714e3da0733fd6b32fb2ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:07.985962 22786 raft_consensus.cc:348] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.986059 22786 raft_consensus.cc:374] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.986107 22786 raft_consensus.cc:729] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550fc6001a224a24bd48931d6321ce05, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.986213 22786 consensus_queue.cc:260] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.986449 22786 ts_tablet_manager.cc:1428] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:07.986929 22784 tablet_bootstrap.cc:492] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: Bootstrap starting.
I20260810 22:39:07.987885 22784 tablet_bootstrap.cc:654] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.988852 22844 tablet_replica.cc:330] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: stopping tablet replica
I20260810 22:39:07.988860 22785 tablet_bootstrap.cc:492] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: Bootstrap starting.
I20260810 22:39:07.989097 22784 tablet_bootstrap.cc:492] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: No bootstrap required, opened a new log
I20260810 22:39:07.989126 22844 raft_consensus.cc:2229] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:07.989171 22784 ts_tablet_manager.cc:1397] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:07.989379 22844 raft_consensus.cc:2258] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:07.989629 22784 raft_consensus.cc:348] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.989723 22784 raft_consensus.cc:374] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:07.989746 22785 tablet_bootstrap.cc:654] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:07.989756 22784 raft_consensus.cc:729] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a65dd6d024f46f1b7733d6b933608fa, State: Initialized, Role: FOLLOWER
I20260810 22:39:07.989864 22784 consensus_queue.cc:260] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:07.990082 22784 ts_tablet_manager.cc:1428] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:07.990689 22844 ts_tablet_manager.cc:1905] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:07.993538 22844 ts_tablet_manager.cc:1918] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:07.993635 22844 log.cc:1196] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/8d309076791e4619a1b6fc41ce56fe67
I20260810 22:39:07.993921 22844 ts_tablet_manager.cc:1939] T 8d309076791e4619a1b6fc41ce56fe67 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting consensus metadata
I20260810 22:39:07.995020 22212 catalog_manager.cc:4841] TS 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507): tablet 8d309076791e4619a1b6fc41ce56fe67 (table test-table [id=0d002353270841168b537d4b18e51982]) successfully deleted
I20260810 22:39:07.996855 22843 ts_tablet_manager.cc:1918] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:07.999373 22843 log.cc:1196] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/8d309076791e4619a1b6fc41ce56fe67
I20260810 22:39:07.999701 22843 ts_tablet_manager.cc:1939] T 8d309076791e4619a1b6fc41ce56fe67 P 550fc6001a224a24bd48931d6321ce05: Deleting consensus metadata
I20260810 22:39:08.001586 22841 ts_tablet_manager.cc:1918] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:08.001690 22841 log.cc:1196] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/8d309076791e4619a1b6fc41ce56fe67
I20260810 22:39:08.001988 22841 ts_tablet_manager.cc:1939] T 8d309076791e4619a1b6fc41ce56fe67 P 8a9c66c20fd54e2f93748204320c5e55: Deleting consensus metadata
I20260810 22:39:08.003136 22212 catalog_manager.cc:4841] TS 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757): tablet 8d309076791e4619a1b6fc41ce56fe67 (table test-table [id=0d002353270841168b537d4b18e51982]) successfully deleted
I20260810 22:39:08.003698 22785 tablet_bootstrap.cc:492] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: No bootstrap required, opened a new log
I20260810 22:39:08.003793 22785 ts_tablet_manager.cc:1397] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260810 22:39:08.003847 22210 catalog_manager.cc:4841] TS 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203): tablet 8d309076791e4619a1b6fc41ce56fe67 (table test-table [id=0d002353270841168b537d4b18e51982]) successfully deleted
I20260810 22:39:08.004240 22785 raft_consensus.cc:348] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:08.004357 22785 raft_consensus.cc:374] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:08.004386 22785 raft_consensus.cc:729] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9c66c20fd54e2f93748204320c5e55, State: Initialized, Role: FOLLOWER
I20260810 22:39:08.004496 22785 consensus_queue.cc:260] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:08.004753 22785 ts_tablet_manager.cc:1428] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:08.229040 22791 raft_consensus.cc:482] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:08.229172 22791 raft_consensus.cc:504] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:08.230123 22791 leader_election.cc:288] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757), 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203)
I20260810 22:39:08.234515 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.230550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:58097 (local address 127.19.202.65:35757)
0810 22:39:08.230659 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:08.230665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:08.230961 (+ 296us) server_negotiation.cc:407] Connection header received
0810 22:39:08.231063 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:08.231066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:08.231114 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:08.231192 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:08.231434 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.232113 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.233484 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.233896 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.234023 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.234078 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:08.234331 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 22:39:08.234403 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:39:08.234640 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.230471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:34701 (local address 127.19.202.66:38203)
0810 22:39:08.230601 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:08.230607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:08.230855 (+ 248us) server_negotiation.cc:407] Connection header received
0810 22:39:08.231502 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:08.231508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:08.231570 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:08.231671 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:08.231967 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.232472 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.233626 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.234041 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.234176 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.234233 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:08.234470 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 22:39:08.234540 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:39:08.234668 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.230372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:38203 (local address 127.19.202.67:34701)
0810 22:39:08.230749 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:08.230767 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:08.230900 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:08.231689 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:08.231692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:08.231711 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:08.231842 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.231847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.232641 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.232644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.233499 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.233509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.234184 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.234188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.234331 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.234350 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 22:39:08.234452 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 22:39:08.234520 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":23,"spinlock_wait_cycles":13184,"thread_start_us":124,"threads_started":1}
I20260810 22:39:08.234951 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.230386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.19.202.67:58097)
0810 22:39:08.230857 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:08.230889 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:08.230970 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:08.231210 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:08.231212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:08.231225 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:08.231330 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.231335 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.232259 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.232264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.233331 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.233342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.234018 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.234022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.234179 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.234218 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 22:39:08.234308 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 22:39:08.234381 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260810 22:39:08.235189 22536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d7417203714e3da0733fd6b32fb2ce" candidate_uuid: "2a65dd6d024f46f1b7733d6b933608fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550fc6001a224a24bd48931d6321ce05" is_pre_election: true
I20260810 22:39:08.235343 22536 raft_consensus.cc:2454] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a65dd6d024f46f1b7733d6b933608fa in term 0.
I20260810 22:39:08.235451 22397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d7417203714e3da0733fd6b32fb2ce" candidate_uuid: "2a65dd6d024f46f1b7733d6b933608fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9c66c20fd54e2f93748204320c5e55" is_pre_election: true
I20260810 22:39:08.235603 22397 raft_consensus.cc:2454] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a65dd6d024f46f1b7733d6b933608fa in term 0.
I20260810 22:39:08.235706 22633 leader_election.cc:302] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a65dd6d024f46f1b7733d6b933608fa, 550fc6001a224a24bd48931d6321ce05; no voters:
I20260810 22:39:08.235958 22791 raft_consensus.cc:2781] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:08.236011 22791 raft_consensus.cc:482] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:08.236035 22791 raft_consensus.cc:3037] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:08.237160 22791 raft_consensus.cc:504] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:08.237530 22791 leader_election.cc:288] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [CANDIDATE]: Term 1 election: Requested vote from peers 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757), 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203)
I20260810 22:39:08.237888 22397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d7417203714e3da0733fd6b32fb2ce" candidate_uuid: "2a65dd6d024f46f1b7733d6b933608fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9c66c20fd54e2f93748204320c5e55"
I20260810 22:39:08.237888 22536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d7417203714e3da0733fd6b32fb2ce" candidate_uuid: "2a65dd6d024f46f1b7733d6b933608fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550fc6001a224a24bd48931d6321ce05"
I20260810 22:39:08.238009 22536 raft_consensus.cc:3037] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:08.238009 22397 raft_consensus.cc:3037] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:08.239115 22397 raft_consensus.cc:2454] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a65dd6d024f46f1b7733d6b933608fa in term 1.
I20260810 22:39:08.239115 22536 raft_consensus.cc:2454] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a65dd6d024f46f1b7733d6b933608fa in term 1.
I20260810 22:39:08.239382 22633 leader_election.cc:302] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a65dd6d024f46f1b7733d6b933608fa, 550fc6001a224a24bd48931d6321ce05; no voters:
I20260810 22:39:08.239575 22791 raft_consensus.cc:2781] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:08.239761 22791 raft_consensus.cc:686] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 1 LEADER]: Becoming Leader. State: Replica: 2a65dd6d024f46f1b7733d6b933608fa, State: Running, Role: LEADER
I20260810 22:39:08.240154 22791 consensus_queue.cc:237] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:08.242297 22227 catalog_manager.cc:5495] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67). New cstate: current_term: 1 leader_uuid: "2a65dd6d024f46f1b7733d6b933608fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } health_report { overall_health: HEALTHY } } }
I20260810 22:39:08.289119 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.285359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.19.202.67:45507)
0810 22:39:08.285478 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:08.285483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:08.285544 (+ 61us) server_negotiation.cc:407] Connection header received
0810 22:39:08.285742 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:08.285745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:08.285806 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:08.285930 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:08.286160 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.286667 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.287509 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.287746 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.287876 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.288122 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:08.288692 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:08.288841 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:08.288922 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 22:39:08.288972 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:39:08.300181 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.296935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:47539 (local address 127.19.202.65:35757)
0810 22:39:08.297101 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:08.297105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:08.297290 (+ 185us) server_negotiation.cc:407] Connection header received
0810 22:39:08.297394 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:08.297396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:08.297439 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:08.297499 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:08.297697 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.298163 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.298935 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.299227 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.299303 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.299716 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:08.299779 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:08.299833 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:08.299974 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 22:39:08.300026 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:39:08.300356 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.296842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.19.202.67:47539)
0810 22:39:08.297226 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:08.297239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:08.297327 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:08.297522 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:08.297524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:08.297529 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:08.297629 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.297633 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.298244 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.298246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.298827 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.298834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.299331 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.299340 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.299497 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.299542 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:08.299856 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:08.299862 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 22:39:08.299913 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 22:39:08.300018 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":163,"threads_started":1}
I20260810 22:39:08.306555 22536 raft_consensus.cc:1261] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Refusing update from remote peer 2a65dd6d024f46f1b7733d6b933608fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:08.306638 22397 raft_consensus.cc:1261] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Refusing update from remote peer 2a65dd6d024f46f1b7733d6b933608fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:08.307116 22850 consensus_queue.cc:1035] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:08.307283 22791 consensus_queue.cc:1035] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:08.323936 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.320586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:47561 (local address 127.19.202.67:45507)
0810 22:39:08.320677 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:08.320681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:08.320704 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:39:08.320869 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:08.320871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:08.320914 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:08.320997 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:08.321226 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.321874 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.322702 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.323035 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.323157 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.323469 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:08.323558 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:08.323633 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:08.323746 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 22:39:08.323786 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:39:08.323954 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:08.320522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45507 (local address 127.19.202.65:47561)
0810 22:39:08.320654 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:08.320669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:08.320743 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:08.321011 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:08.321014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:08.321022 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:08.321118 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.321124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.321985 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.321989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.322582 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:08.322591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:08.323148 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:08.323172 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:08.323263 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:08.323299 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:08.323665 (+ 366us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:08.323672 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 22:39:08.323740 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 22:39:08.323824 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 22:39:08.324731 22817 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 22:39:09.882526 22227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:09.882676 22227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:09.884873 22227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Sending DeleteTablet for 3 replicas of tablet 72d7417203714e3da0733fd6b32fb2ce
I20260810 22:39:09.885341 22376 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d7417203714e3da0733fd6b32fb2ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:09 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:34250
I20260810 22:39:09.885418 22687 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d7417203714e3da0733fd6b32fb2ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:09 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:33836
I20260810 22:39:09.885398 22523 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d7417203714e3da0733fd6b32fb2ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:09 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:54602
I20260810 22:39:09.885701 22884 tablet_replica.cc:330] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: stopping tablet replica
I20260810 22:39:09.885831 22884 raft_consensus.cc:2229] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:09.885728 22885 tablet_replica.cc:330] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: stopping tablet replica
I20260810 22:39:09.885877 22883 tablet_replica.cc:330] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: stopping tablet replica
I20260810 22:39:09.885975 22885 raft_consensus.cc:2229] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:09.886099 22885 raft_consensus.cc:2258] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:09.886137 22884 raft_consensus.cc:2258] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:09.886193 22883 raft_consensus.cc:2229] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:09.886396 22883 raft_consensus.cc:2258] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:09.886802 22884 ts_tablet_manager.cc:1905] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:09.886814 22885 ts_tablet_manager.cc:1905] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:09.887149 22883 ts_tablet_manager.cc:1905] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:09.889780 22884 ts_tablet_manager.cc:1918] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:39:09.889871 22884 log.cc:1196] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/72d7417203714e3da0733fd6b32fb2ce
I20260810 22:39:09.890187 22884 ts_tablet_manager.cc:1939] T 72d7417203714e3da0733fd6b32fb2ce P 2a65dd6d024f46f1b7733d6b933608fa: Deleting consensus metadata
I20260810 22:39:09.890479 22883 ts_tablet_manager.cc:1918] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:39:09.890627 22883 log.cc:1196] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/72d7417203714e3da0733fd6b32fb2ce
I20260810 22:39:09.890780 22885 ts_tablet_manager.cc:1918] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:39:09.890880 22212 catalog_manager.cc:4841] TS 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507): tablet 72d7417203714e3da0733fd6b32fb2ce (table test-table [id=29d05cb92a214c2daefcb3e3e301c173]) successfully deleted
I20260810 22:39:09.890955 22885 log.cc:1196] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/72d7417203714e3da0733fd6b32fb2ce
I20260810 22:39:09.891218 22883 ts_tablet_manager.cc:1939] T 72d7417203714e3da0733fd6b32fb2ce P 8a9c66c20fd54e2f93748204320c5e55: Deleting consensus metadata
I20260810 22:39:09.891364 22885 ts_tablet_manager.cc:1939] T 72d7417203714e3da0733fd6b32fb2ce P 550fc6001a224a24bd48931d6321ce05: Deleting consensus metadata
I20260810 22:39:09.891877 22212 catalog_manager.cc:4841] TS 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757): tablet 72d7417203714e3da0733fd6b32fb2ce (table test-table [id=29d05cb92a214c2daefcb3e3e301c173]) successfully deleted
I20260810 22:39:09.892352 22210 catalog_manager.cc:4841] TS 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203): tablet 72d7417203714e3da0733fd6b32fb2ce (table test-table [id=29d05cb92a214c2daefcb3e3e301c173]) successfully deleted
Aug 10 22:39:09 dist-test-slave-1x4g krb5kdc[22177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:39:09 dist-test-slave-1x4g krb5kdc[22177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:09.922487 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:09.909023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.19.202.126:33883)
0810 22:39:09.909322 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:09.909328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:09.909342 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:39:09.909432 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:09.909434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:09.909474 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:09.909549 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:09.910196 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.910737 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.911493 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.911704 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.911776 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:09.919736 (+ 7960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:09.919740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:09.919742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:09.919765 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:09.921045 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:09.921382 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:09.921384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:09.921385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:09.921455 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:09.921667 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:09.921669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:09.921670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:09.921828 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:09.921922 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:09.922157 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 22:39:09.922253 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":120,"threads_started":1}
I20260810 22:39:09.925086 22227 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:09.925482 22227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:09.929981 22376 tablet_service.cc:1467] Processing CreateTablet for tablet dfd9a6b088324ebfbe2e5784404dd944 (DEFAULT_TABLE table=test-table [id=123b28bad4044dbda9d471fbb8d2fdec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:09.930079 22523 tablet_service.cc:1467] Processing CreateTablet for tablet dfd9a6b088324ebfbe2e5784404dd944 (DEFAULT_TABLE table=test-table [id=123b28bad4044dbda9d471fbb8d2fdec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:09.930130 22687 tablet_service.cc:1467] Processing CreateTablet for tablet dfd9a6b088324ebfbe2e5784404dd944 (DEFAULT_TABLE table=test-table [id=123b28bad4044dbda9d471fbb8d2fdec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:09.930310 22376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd9a6b088324ebfbe2e5784404dd944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:09.930352 22687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd9a6b088324ebfbe2e5784404dd944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:09.930352 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd9a6b088324ebfbe2e5784404dd944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:09.932608 22895 tablet_bootstrap.cc:492] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: Bootstrap starting.
I20260810 22:39:09.933081 22897 tablet_bootstrap.cc:492] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: Bootstrap starting.
I20260810 22:39:09.933341 22896 tablet_bootstrap.cc:492] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: Bootstrap starting.
I20260810 22:39:09.933739 22895 tablet_bootstrap.cc:654] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:09.934054 22897 tablet_bootstrap.cc:654] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:09.934260 22896 tablet_bootstrap.cc:654] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:09.934965 22895 tablet_bootstrap.cc:492] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: No bootstrap required, opened a new log
I20260810 22:39:09.935075 22895 ts_tablet_manager.cc:1397] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:09.935222 22897 tablet_bootstrap.cc:492] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: No bootstrap required, opened a new log
I20260810 22:39:09.935316 22897 ts_tablet_manager.cc:1397] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:09.935568 22895 raft_consensus.cc:348] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.935689 22895 raft_consensus.cc:374] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:09.935729 22895 raft_consensus.cc:729] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9c66c20fd54e2f93748204320c5e55, State: Initialized, Role: FOLLOWER
I20260810 22:39:09.935750 22897 raft_consensus.cc:348] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.935865 22897 raft_consensus.cc:374] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:09.935855 22895 consensus_queue.cc:260] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.935912 22897 raft_consensus.cc:729] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a65dd6d024f46f1b7733d6b933608fa, State: Initialized, Role: FOLLOWER
I20260810 22:39:09.936065 22897 consensus_queue.cc:260] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.936342 22897 ts_tablet_manager.cc:1428] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 22:39:09.936621 22895 ts_tablet_manager.cc:1428] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 22:39:09.936924 22896 tablet_bootstrap.cc:492] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: No bootstrap required, opened a new log
I20260810 22:39:09.937016 22896 ts_tablet_manager.cc:1397] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 22:39:09.937448 22896 raft_consensus.cc:348] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.937611 22896 raft_consensus.cc:374] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:09.937660 22896 raft_consensus.cc:729] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550fc6001a224a24bd48931d6321ce05, State: Initialized, Role: FOLLOWER
I20260810 22:39:09.937825 22896 consensus_queue.cc:260] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.938278 22896 ts_tablet_manager.cc:1428] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:09.959084 22878 raft_consensus.cc:482] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:09.959170 22878 raft_consensus.cc:504] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.959564 22878 leader_election.cc:288] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:09.959910 22554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd9a6b088324ebfbe2e5784404dd944" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550fc6001a224a24bd48931d6321ce05" is_pre_election: true
I20260810 22:39:09.959909 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd9a6b088324ebfbe2e5784404dd944" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa" is_pre_election: true
I20260810 22:39:09.960063 22708 raft_consensus.cc:2454] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 0.
I20260810 22:39:09.960063 22554 raft_consensus.cc:2454] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 0.
I20260810 22:39:09.960366 22326 leader_election.cc:302] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a65dd6d024f46f1b7733d6b933608fa, 8a9c66c20fd54e2f93748204320c5e55; no voters:
I20260810 22:39:09.960517 22878 raft_consensus.cc:2781] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:09.960573 22878 raft_consensus.cc:482] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:09.960639 22878 raft_consensus.cc:3037] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:09.961668 22878 raft_consensus.cc:504] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.961999 22878 leader_election.cc:288] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 election: Requested vote from peers 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:09.962270 22554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd9a6b088324ebfbe2e5784404dd944" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550fc6001a224a24bd48931d6321ce05"
I20260810 22:39:09.962316 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd9a6b088324ebfbe2e5784404dd944" candidate_uuid: "8a9c66c20fd54e2f93748204320c5e55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa"
I20260810 22:39:09.962407 22554 raft_consensus.cc:3037] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:09.962438 22708 raft_consensus.cc:3037] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:09.963387 22554 raft_consensus.cc:2454] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 1.
I20260810 22:39:09.963439 22708 raft_consensus.cc:2454] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a9c66c20fd54e2f93748204320c5e55 in term 1.
I20260810 22:39:09.963703 22324 leader_election.cc:302] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550fc6001a224a24bd48931d6321ce05, 8a9c66c20fd54e2f93748204320c5e55; no voters:
I20260810 22:39:09.963908 22878 raft_consensus.cc:2781] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:09.963992 22878 raft_consensus.cc:686] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 LEADER]: Becoming Leader. State: Replica: 8a9c66c20fd54e2f93748204320c5e55, State: Running, Role: LEADER
I20260810 22:39:09.964115 22878 consensus_queue.cc:237] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:09.965700 22223 catalog_manager.cc:5495] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "8a9c66c20fd54e2f93748204320c5e55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:09.984072 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:09.980859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34286 (local address 127.19.202.65:35757)
0810 22:39:09.981130 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:09.981139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:09.981150 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:39:09.981192 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:09.981194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:09.981251 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:09.981346 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:09.981671 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.982157 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.983023 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.983265 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.983373 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:09.983652 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:09.983721 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:09.983792 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:09.983875 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 22:39:09.983931 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
I20260810 22:39:09.992076 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:09.989454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:41331 (local address 127.19.202.65:35757)
0810 22:39:09.989553 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:09.989555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:09.989675 (+ 120us) server_negotiation.cc:407] Connection header received
0810 22:39:09.989747 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:09.989749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:09.989781 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:09.989834 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:09.990047 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.990494 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.991209 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.991397 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.991470 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:09.991702 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:09.991770 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:09.991820 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:09.991896 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 22:39:09.991954 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":41}
I20260810 22:39:09.992096 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:09.989297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.19.202.65:41331)
0810 22:39:09.989599 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:09.989610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:09.989693 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:09.989858 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:09.989860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:09.989865 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:09.989987 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:09.989990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.990577 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.990578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:09.991109 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:09.991116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:09.991491 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:09.991493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:09.991567 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:09.991592 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:09.991840 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:09.991845 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:39:09.991890 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 22:39:09.991973 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":29,"thread_start_us":94,"threads_started":1}
I20260810 22:39:09.997623 22708 raft_consensus.cc:1261] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Refusing update from remote peer 8a9c66c20fd54e2f93748204320c5e55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:09.997754 22554 raft_consensus.cc:1261] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Refusing update from remote peer 8a9c66c20fd54e2f93748204320c5e55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:09.998071 22878 consensus_queue.cc:1035] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:09.998266 22833 consensus_queue.cc:1035] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:10.000607 22859 mvcc.cc:205] Tried to move back new op lower bound from 7317100748787113984 to 7317100748654231552. Current Snapshot: MvccSnapshot[applied={T|T < 7317100748787113984}]
I20260810 22:39:10.030450 22227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:10.030642 22227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:10.032680 22227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Sending DeleteTablet for 3 replicas of tablet dfd9a6b088324ebfbe2e5784404dd944
I20260810 22:39:10.033305 22376 tablet_service.cc:1514] Processing DeleteTablet for tablet dfd9a6b088324ebfbe2e5784404dd944 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:10 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:34250
I20260810 22:39:10.033329 22523 tablet_service.cc:1514] Processing DeleteTablet for tablet dfd9a6b088324ebfbe2e5784404dd944 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:10 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:54602
I20260810 22:39:10.033489 22885 tablet_replica.cc:330] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: stopping tablet replica
I20260810 22:39:10.033490 22883 tablet_replica.cc:330] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: stopping tablet replica
I20260810 22:39:10.033612 22883 raft_consensus.cc:2229] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:10.033612 22885 raft_consensus.cc:2229] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:10.033715 22885 raft_consensus.cc:2258] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:10.033895 22883 raft_consensus.cc:2258] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:10.034477 22885 ts_tablet_manager.cc:1905] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:10.034560 22883 ts_tablet_manager.cc:1905] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:10.034744 22687 tablet_service.cc:1514] Processing DeleteTablet for tablet dfd9a6b088324ebfbe2e5784404dd944 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:10 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:33836
I20260810 22:39:10.034919 22884 tablet_replica.cc:330] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: stopping tablet replica
I20260810 22:39:10.035017 22884 raft_consensus.cc:2229] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:10.035123 22884 raft_consensus.cc:2258] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:10.035353 22227 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:10.035750 22227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:10.037297 22885 ts_tablet_manager.cc:1918] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:10.037350 22885 log.cc:1196] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/dfd9a6b088324ebfbe2e5784404dd944
I20260810 22:39:10.037402 22883 ts_tablet_manager.cc:1918] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:10.037473 22883 log.cc:1196] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/dfd9a6b088324ebfbe2e5784404dd944
I20260810 22:39:10.037544 22885 ts_tablet_manager.cc:1939] T dfd9a6b088324ebfbe2e5784404dd944 P 550fc6001a224a24bd48931d6321ce05: Deleting consensus metadata
I20260810 22:39:10.037737 22883 ts_tablet_manager.cc:1939] T dfd9a6b088324ebfbe2e5784404dd944 P 8a9c66c20fd54e2f93748204320c5e55: Deleting consensus metadata
I20260810 22:39:10.038200 22210 catalog_manager.cc:4841] TS 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203): tablet dfd9a6b088324ebfbe2e5784404dd944 (table test-table [id=123b28bad4044dbda9d471fbb8d2fdec]) successfully deleted
I20260810 22:39:10.038451 22212 catalog_manager.cc:4841] TS 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757): tablet dfd9a6b088324ebfbe2e5784404dd944 (table test-table [id=123b28bad4044dbda9d471fbb8d2fdec]) successfully deleted
I20260810 22:39:10.038851 22884 ts_tablet_manager.cc:1905] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:10.040978 22884 ts_tablet_manager.cc:1918] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:10.041065 22884 log.cc:1196] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/dfd9a6b088324ebfbe2e5784404dd944
I20260810 22:39:10.041287 22884 ts_tablet_manager.cc:1939] T dfd9a6b088324ebfbe2e5784404dd944 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting consensus metadata
I20260810 22:39:10.041901 22212 catalog_manager.cc:4841] TS 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507): tablet dfd9a6b088324ebfbe2e5784404dd944 (table test-table [id=123b28bad4044dbda9d471fbb8d2fdec]) successfully deleted
I20260810 22:39:10.044952 22376 tablet_service.cc:1467] Processing CreateTablet for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 (DEFAULT_TABLE table=test-table [id=aca0a105bf5b4469bca3c43bd3c4537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:10.045086 22523 tablet_service.cc:1467] Processing CreateTablet for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 (DEFAULT_TABLE table=test-table [id=aca0a105bf5b4469bca3c43bd3c4537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:10.045194 22376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:10.045349 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:10.047144 22896 tablet_bootstrap.cc:492] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: Bootstrap starting.
I20260810 22:39:10.047178 22895 tablet_bootstrap.cc:492] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55: Bootstrap starting.
I20260810 22:39:10.047335 22687 tablet_service.cc:1467] Processing CreateTablet for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 (DEFAULT_TABLE table=test-table [id=aca0a105bf5b4469bca3c43bd3c4537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:10.047576 22687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:10.048122 22895 tablet_bootstrap.cc:654] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:10.048154 22896 tablet_bootstrap.cc:654] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:10.049341 22896 tablet_bootstrap.cc:492] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: No bootstrap required, opened a new log
I20260810 22:39:10.049430 22896 ts_tablet_manager.cc:1397] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:10.049924 22896 raft_consensus.cc:348] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.050051 22896 raft_consensus.cc:374] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:10.050087 22896 raft_consensus.cc:729] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550fc6001a224a24bd48931d6321ce05, State: Initialized, Role: FOLLOWER
I20260810 22:39:10.050136 22897 tablet_bootstrap.cc:492] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: Bootstrap starting.
I20260810 22:39:10.050230 22896 consensus_queue.cc:260] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.050493 22896 ts_tablet_manager.cc:1428] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:10.051191 22897 tablet_bootstrap.cc:654] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:10.051340 22895 tablet_bootstrap.cc:492] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55: No bootstrap required, opened a new log
I20260810 22:39:10.051424 22895 ts_tablet_manager.cc:1397] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 22:39:10.051767 22895 raft_consensus.cc:348] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.051875 22895 raft_consensus.cc:374] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:10.051911 22895 raft_consensus.cc:729] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9c66c20fd54e2f93748204320c5e55, State: Initialized, Role: FOLLOWER
I20260810 22:39:10.052014 22895 consensus_queue.cc:260] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.052256 22895 ts_tablet_manager.cc:1428] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 22:39:10.053385 22897 tablet_bootstrap.cc:492] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: No bootstrap required, opened a new log
I20260810 22:39:10.053496 22897 ts_tablet_manager.cc:1397] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:10.053972 22897 raft_consensus.cc:348] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.054082 22897 raft_consensus.cc:374] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:10.054122 22897 raft_consensus.cc:729] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a65dd6d024f46f1b7733d6b933608fa, State: Initialized, Role: FOLLOWER
I20260810 22:39:10.054261 22897 consensus_queue.cc:260] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.054782 22897 ts_tablet_manager.cc:1428] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:10.219796 22901 raft_consensus.cc:482] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:10.220039 22901 raft_consensus.cc:504] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.220525 22901 leader_election.cc:288] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:10.221057 22396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a67ecf9262949a3ad3e3ce7a4c05d50" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9c66c20fd54e2f93748204320c5e55" is_pre_election: true
I20260810 22:39:10.221109 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a67ecf9262949a3ad3e3ce7a4c05d50" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa" is_pre_election: true
I20260810 22:39:10.221190 22396 raft_consensus.cc:2454] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 0.
I20260810 22:39:10.221355 22708 raft_consensus.cc:2454] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 0.
I20260810 22:39:10.221596 22480 leader_election.cc:302] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550fc6001a224a24bd48931d6321ce05, 8a9c66c20fd54e2f93748204320c5e55; no voters:
I20260810 22:39:10.221765 22901 raft_consensus.cc:2781] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:10.221827 22901 raft_consensus.cc:482] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:10.221868 22901 raft_consensus.cc:3037] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:10.223027 22901 raft_consensus.cc:504] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.223332 22901 leader_election.cc:288] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 election: Requested vote from peers 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757), 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507)
I20260810 22:39:10.223688 22396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a67ecf9262949a3ad3e3ce7a4c05d50" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9c66c20fd54e2f93748204320c5e55"
I20260810 22:39:10.223733 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a67ecf9262949a3ad3e3ce7a4c05d50" candidate_uuid: "550fc6001a224a24bd48931d6321ce05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a65dd6d024f46f1b7733d6b933608fa"
I20260810 22:39:10.223847 22396 raft_consensus.cc:3037] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:10.223918 22708 raft_consensus.cc:3037] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:10.224903 22396 raft_consensus.cc:2454] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 1.
I20260810 22:39:10.225180 22708 raft_consensus.cc:2454] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 550fc6001a224a24bd48931d6321ce05 in term 1.
I20260810 22:39:10.225229 22480 leader_election.cc:302] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550fc6001a224a24bd48931d6321ce05, 8a9c66c20fd54e2f93748204320c5e55; no voters:
I20260810 22:39:10.225409 22901 raft_consensus.cc:2781] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:10.225626 22901 raft_consensus.cc:686] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 1 LEADER]: Becoming Leader. State: Replica: 550fc6001a224a24bd48931d6321ce05, State: Running, Role: LEADER
I20260810 22:39:10.225800 22901 consensus_queue.cc:237] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } }
I20260810 22:39:10.227550 22227 catalog_manager.cc:5495] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 550fc6001a224a24bd48931d6321ce05 (127.19.202.66). New cstate: current_term: 1 leader_uuid: "550fc6001a224a24bd48931d6321ce05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "550fc6001a224a24bd48931d6321ce05" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:10.245270 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:10.242133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.19.202.66:38203)
0810 22:39:10.242605 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:10.242613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:10.242625 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:39:10.242663 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:10.242665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:10.242707 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:10.242783 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:10.243059 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:10.243585 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:10.244347 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:10.244543 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:10.244615 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:10.244847 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:10.244991 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:10.245048 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:10.245122 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 22:39:10.245165 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":108,"threads_started":1}
I20260810 22:39:10.250847 22708 raft_consensus.cc:1261] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Refusing update from remote peer 550fc6001a224a24bd48931d6321ce05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:10.250964 22396 raft_consensus.cc:1261] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Refusing update from remote peer 550fc6001a224a24bd48931d6321ce05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:10.251201 22901 consensus_queue.cc:1035] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a65dd6d024f46f1b7733d6b933608fa" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:10.251353 22916 consensus_queue.cc:1035] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a9c66c20fd54e2f93748204320c5e55" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 22:39:10.265802 22863 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 22:39:11.826814 22227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:11.827090 22227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:11.829402 22227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee616e7f86c043958839fea519ec3959: Sending DeleteTablet for 3 replicas of tablet 1a67ecf9262949a3ad3e3ce7a4c05d50
I20260810 22:39:11.829926 22523 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:11 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:54602
I20260810 22:39:11.829926 22376 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:11 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:34250
I20260810 22:39:11.830041 22687 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:11 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:33836
I20260810 22:39:11.830320 22944 tablet_replica.cc:330] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: stopping tablet replica
I20260810 22:39:11.830428 22945 tablet_replica.cc:330] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55: stopping tablet replica
I20260810 22:39:11.830459 22944 raft_consensus.cc:2229] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:11.830538 22945 raft_consensus.cc:2229] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:11.830643 22945 raft_consensus.cc:2258] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 8a9c66c20fd54e2f93748204320c5e55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:11.830646 22944 raft_consensus.cc:2258] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:11.830741 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 22306
I20260810 22:39:11.833478 22944 ts_tablet_manager.cc:1905] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:11.836004 22946 tablet_replica.cc:330] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: stopping tablet replica
I20260810 22:39:11.836161 22946 raft_consensus.cc:2229] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:11.836347 22946 raft_consensus.cc:2258] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:11.837005 22946 ts_tablet_manager.cc:1905] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:11.837361 22944 ts_tablet_manager.cc:1918] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:39:11.837462 22944 log.cc:1196] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/1a67ecf9262949a3ad3e3ce7a4c05d50
I20260810 22:39:11.837786 22944 ts_tablet_manager.cc:1939] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 550fc6001a224a24bd48931d6321ce05: Deleting consensus metadata
I20260810 22:39:11.838652 22210 catalog_manager.cc:4841] TS 550fc6001a224a24bd48931d6321ce05 (127.19.202.66:38203): tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 (table test-table [id=aca0a105bf5b4469bca3c43bd3c4537c]) successfully deleted
I20260810 22:39:11.839368 22946 ts_tablet_manager.cc:1918] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:39:11.839458 22946 log.cc:1196] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/1a67ecf9262949a3ad3e3ce7a4c05d50
I20260810 22:39:11.839725 22946 ts_tablet_manager.cc:1939] T 1a67ecf9262949a3ad3e3ce7a4c05d50 P 2a65dd6d024f46f1b7733d6b933608fa: Deleting consensus metadata
I20260810 22:39:11.840337 22212 catalog_manager.cc:4841] TS 2a65dd6d024f46f1b7733d6b933608fa (127.19.202.67:45507): tablet 1a67ecf9262949a3ad3e3ce7a4c05d50 (table test-table [id=aca0a105bf5b4469bca3c43bd3c4537c]) successfully deleted
W20260810 22:39:11.845171 22212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:35757 (error 108)
I20260810 22:39:11.845307 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 22460
I20260810 22:39:11.848241 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:11.847276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35757 (local address 127.0.0.1:34298)
0810 22:39:11.847995 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:11.848109 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":97,"threads_started":1}
W20260810 22:39:11.848373 22212 catalog_manager.cc:4584] TS 8a9c66c20fd54e2f93748204320c5e55 (127.19.202.65:35757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a67ecf9262949a3ad3e3ce7a4c05d50: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35757: connect: Connection refused (error 111)
I20260810 22:39:11.854286 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 22615
I20260810 22:39:11.862927 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 22192
2026-08-10T22:39:11Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6521 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:11 dist-test-slave-1x4g krb5kdc[22952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:11 dist-test-slave-1x4g krb5kdc[22952](info): set up 2 sockets
Aug 10 22:39:11 dist-test-slave-1x4g krb5kdc[22952](info): commencing operation
krb5kdc: starting...
W20260810 22:39:12.967759 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:13.086941 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:42395
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:43697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:42395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:13.196324 22968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:13.196578 22968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:13.196617 22968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:13.200332 22968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:13.200380 22968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:13.200399 22968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:13.200414 22968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:13.200428 22968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:13.205430 22968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:43697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:42395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:13.206357 22968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:13.207278 22968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:13.216071 22974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:13.216053 22975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:13.216109 22968 server_base.cc:1034] running on GCE node
W20260810 22:39:13.216053 22977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:13.216995 22968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:13.218096 22968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:13.219316 22968 hybrid_clock.cc:648] HybridClock initialized: now 1786401553219289 us; error 42 us; skew 500 ppm
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:13.222154 22968 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:13.223071 22968 webserver.cc:460] Webserver started at http://127.19.202.126:41841/ using document root <none> and password file <none>
I20260810 22:39:13.223640 22968 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:13.223742 22968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:13.224007 22968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:13.226109 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "58a3f304f8e04ca5b6d4673f4ed42535"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "c57dd4353ef28b707a14b0a238eb19de"
server_key_iv: "2ca2b5551856bc3d3c615a03e6265532"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.226701 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "58a3f304f8e04ca5b6d4673f4ed42535"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "c57dd4353ef28b707a14b0a238eb19de"
server_key_iv: "2ca2b5551856bc3d3c615a03e6265532"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.230649 22968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 22:39:13.233174 22984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.234405 22968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260810 22:39:13.234550 22968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "58a3f304f8e04ca5b6d4673f4ed42535"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "c57dd4353ef28b707a14b0a238eb19de"
server_key_iv: "2ca2b5551856bc3d3c615a03e6265532"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.234664 22968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:13.249748 22968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:13.252606 22968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:13.252851 22968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:13.260540 22968 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:42395
I20260810 22:39:13.260587 23040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:42395 every 8 connection(s)
I20260810 22:39:13.261615 22968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:13.262470 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 22968
I20260810 22:39:13.262607 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:13.262903 20265 external_mini_cluster.cc:1428] Setting key ef57fe1f14d8a15a503e9a8812c133f4
I20260810 22:39:13.265038 23041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:13.270471 23041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535: Bootstrap starting.
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:13.272950 23041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:13.273782 23041 log.cc:824] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:13.275956 23041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535: No bootstrap required, opened a new log
I20260810 22:39:13.277367 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:13.264330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.19.202.126:42395)
0810 22:39:13.264804 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:13.264818 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:13.264854 (+ 36us) server_negotiation.cc:407] Connection header received
0810 22:39:13.265597 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:13.265614 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:13.265941 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:13.266293 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:13.267146 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.268137 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.269050 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.269500 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.269711 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:13.272461 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:13.272466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:13.272479 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:13.272507 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:13.274441 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.274944 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.274949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.274953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.275064 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.275353 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.275355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.275356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.275759 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:13.275909 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:13.276255 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 22:39:13.276580 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":141,"threads_started":1}
I20260810 22:39:13.279518 23041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42395 } }
I20260810 22:39:13.279737 23041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:13.279814 23041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a3f304f8e04ca5b6d4673f4ed42535, State: Initialized, Role: FOLLOWER
I20260810 22:39:13.280244 23041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42395 } }
I20260810 22:39:13.280421 23041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:13.280493 23041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:13.280577 23041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:13.281451 23041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42395 } }
I20260810 22:39:13.281781 23041 leader_election.cc:302] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58a3f304f8e04ca5b6d4673f4ed42535; no voters:
I20260810 22:39:13.282071 23041 leader_election.cc:288] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:13.282166 23047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:13.282455 23047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [term 1 LEADER]: Becoming Leader. State: Replica: 58a3f304f8e04ca5b6d4673f4ed42535, State: Running, Role: LEADER
I20260810 22:39:13.282819 23047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42395 } }
I20260810 22:39:13.283239 23041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:13.284469 23048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42395 } } }
I20260810 22:39:13.284590 23048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:13.284946 23050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a3f304f8e04ca5b6d4673f4ed42535. Latest consensus state: current_term: 1 leader_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3f304f8e04ca5b6d4673f4ed42535" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42395 } } }
I20260810 22:39:13.285079 23050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:13.285823 23057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:13.288743 23057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:13.293833 23057 catalog_manager.cc:1347] Generated new cluster ID: 2454c7eb6faf4a5193d8d44dabe9203a
I20260810 22:39:13.293895 23057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:13.303808 23057 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:13.304992 23057 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:13.312291 23057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58a3f304f8e04ca5b6d4673f4ed42535: Generated new TSK 0
I20260810 22:39:13.313097 23057 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:13.351390 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:42395
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:43697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:13.465864 23069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:13.466104 23069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:13.466145 23069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:13.470193 23069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:13.470252 23069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:13.470346 23069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:13.475275 23069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:43697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:13.476219 23069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:13.477350 23069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:39:13.487035 23069 server_base.cc:1034] running on GCE node
W20260810 22:39:13.486989 23076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:13.486990 23075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:13.487007 23078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:13.487609 23069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:13.488176 23069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:13.489357 23069 hybrid_clock.cc:648] HybridClock initialized: now 1786401553489342 us; error 28 us; skew 500 ppm
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:13.492034 23069 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:13.492893 23069 webserver.cc:460] Webserver started at http://127.19.202.65:46835/ using document root <none> and password file <none>
I20260810 22:39:13.493435 23069 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:13.493525 23069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:13.493727 23069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:13.495645 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "2fc7790321684ffa8173f64ac964cb29"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "a5b39071346473672fa1d487098254de"
server_key_iv: "049a479af4eb29b2c163ce324aa8e5d1"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.496172 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "2fc7790321684ffa8173f64ac964cb29"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "a5b39071346473672fa1d487098254de"
server_key_iv: "049a479af4eb29b2c163ce324aa8e5d1"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.499751 23069 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 22:39:13.502032 23086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.503176 23069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:13.503309 23069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "2fc7790321684ffa8173f64ac964cb29"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "a5b39071346473672fa1d487098254de"
server_key_iv: "049a479af4eb29b2c163ce324aa8e5d1"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.503422 23069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:13.517211 23069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:13.519647 23069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:13.519821 23069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:13.520313 23069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:13.521090 23069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:13.521154 23069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.521216 23069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:13.521256 23069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.529830 23069 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:40671
I20260810 22:39:13.529871 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:40671 every 8 connection(s)
I20260810 22:39:13.530681 23069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:13.537644 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 23069
I20260810 22:39:13.537761 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:13.538084 20265 external_mini_cluster.cc:1428] Setting key 8f99ba5b1e4e594d058bfead23a87ef4
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:13.545261 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:13.532344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:49895 (local address 127.19.202.126:42395)
0810 22:39:13.532476 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:13.532486 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:13.533349 (+ 863us) server_negotiation.cc:407] Connection header received
0810 22:39:13.534661 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:13.534664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:13.534711 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:13.534784 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:13.535879 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.536437 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.538032 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.538394 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.538509 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:13.541335 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:13.541338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:13.541340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:13.541367 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:13.543011 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.543542 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.543548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.543553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.543648 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.543992 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.543998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.544002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.544246 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:13.544374 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:13.544937 (+ 563us) server_negotiation.cc:299] Negotiation successful
0810 22:39:13.545071 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:39:13.545902 23213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:13.532695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:42395 (local address 127.19.202.65:49895)
0810 22:39:13.533189 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:13.533231 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:13.534471 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:13.534953 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:13.534959 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:13.535306 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:13.535745 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:13.535758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.536544 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.536547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:13.537905 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:13.537919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.538550 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.538554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:13.538785 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:13.539380 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:13.539396 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:13.541175 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:13.543142 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:13.543146 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:13.543156 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:13.543425 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:13.543743 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:13.543745 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:13.543746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:13.543891 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:13.544353 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:13.544357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:13.544570 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 22:39:13.544854 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 22:39:13.545103 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":114,"threads_started":1}
I20260810 22:39:13.547183 23211 heartbeater.cc:347] Connected to a master server at 127.19.202.126:42395
I20260810 22:39:13.547430 23211 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:13.547981 23211 heartbeater.cc:511] Master 127.19.202.126:42395 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:39:13.549599 23004 ts_manager.cc:194] Registered new tserver with Master: 2fc7790321684ffa8173f64ac964cb29 (127.19.202.65:40671)
I20260810 22:39:13.550701 23004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:49895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:13.565763 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:42395
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:43697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:13.673576 23216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:13.673799 23216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:13.673837 23216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:13.677717 23216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:13.677775 23216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:13.677898 23216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:13.682821 23216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:43697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:13.683838 23216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:13.684865 23216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:13.694104 23223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:13.694154 23221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:13.694187 23225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:13.694541 23216 server_base.cc:1034] running on GCE node
I20260810 22:39:13.694931 23216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:13.695489 23216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:13.696723 23216 hybrid_clock.cc:648] HybridClock initialized: now 1786401553696712 us; error 25 us; skew 500 ppm
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:13.699334 23216 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:13.700160 23216 webserver.cc:460] Webserver started at http://127.19.202.66:45075/ using document root <none> and password file <none>
I20260810 22:39:13.700683 23216 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:13.700770 23216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:13.701056 23216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:13.703023 23216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "9f8663d3701f4db08c3838b09377f98c"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "100bd89af67a2c584c59299f77b27ff4"
server_key_iv: "743a20ce6caeaadda3986aea72766ff8"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.703583 23216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "9f8663d3701f4db08c3838b09377f98c"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "100bd89af67a2c584c59299f77b27ff4"
server_key_iv: "743a20ce6caeaadda3986aea72766ff8"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.707154 23216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:39:13.709393 23233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.710565 23216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:13.710698 23216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "9f8663d3701f4db08c3838b09377f98c"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "100bd89af67a2c584c59299f77b27ff4"
server_key_iv: "743a20ce6caeaadda3986aea72766ff8"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.710813 23216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:13.736442 23216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:13.739554 23216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:13.739755 23216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:13.740350 23216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:13.741197 23216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:13.741276 23216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.741349 23216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:13.741415 23216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.750119 23216 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:41261
I20260810 22:39:13.750154 23361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:41261 every 8 connection(s)
I20260810 22:39:13.751044 23216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:13.751277 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 23216
I20260810 22:39:13.751384 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:13.751713 20265 external_mini_cluster.cc:1428] Setting key 3a21f2b0dc500672667303b55d9855de
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
I20260810 22:39:13.766350 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:13.753027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:41597 (local address 127.19.202.126:42395)
0810 22:39:13.753166 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:13.753170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:13.754659 (+ 1489us) server_negotiation.cc:407] Connection header received
0810 22:39:13.755549 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:13.755552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:13.755606 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:13.755700 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:13.757058 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.757581 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.759443 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.759660 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.759741 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:13.762500 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:13.762508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:13.762511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:13.762533 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:13.764158 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.764675 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.764679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.764681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.764764 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.765161 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.765165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.765167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.765379 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:13.765497 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:13.766057 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 22:39:13.766180 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:39:13.767159 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:13.753472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:42395 (local address 127.19.202.66:41597)
0810 22:39:13.754493 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:13.754545 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:13.755352 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:13.755903 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:13.755909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:13.756287 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:13.756900 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:13.756924 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.757719 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.757725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:13.759303 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:13.759319 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.759823 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.759830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:13.760062 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:13.760676 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:13.760699 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:13.762323 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:13.764290 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:13.764296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:13.764311 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:13.764569 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:13.764880 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:13.764884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:13.764886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:13.765031 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:13.765546 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:13.765551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:13.765810 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 22:39:13.766015 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 22:39:13.766319 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":672,"threads_started":1}
I20260810 22:39:13.768443 23362 heartbeater.cc:347] Connected to a master server at 127.19.202.126:42395
I20260810 22:39:13.768720 23362 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:13.769343 23362 heartbeater.cc:511] Master 127.19.202.126:42395 requested a full tablet report, sending...
I20260810 22:39:13.770558 23004 ts_manager.cc:194] Registered new tserver with Master: 9f8663d3701f4db08c3838b09377f98c (127.19.202.66:41261)
I20260810 22:39:13.771446 23004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:41597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:13.781564 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:42395
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:43697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:13.891462 23369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:13.891681 23369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:13.891719 23369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:13.895545 23369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:13.895598 23369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:13.895650 23369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:13.900449 23369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:43697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:13.901433 23369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:13.902328 23369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:13.911865 23377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:13.912036 23369 server_base.cc:1034] running on GCE node
W20260810 22:39:13.911865 23374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:13.911955 23375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:13.912551 23369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:13.913228 23369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:13.914407 23369 hybrid_clock.cc:648] HybridClock initialized: now 1786401553914378 us; error 43 us; skew 500 ppm
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:13.917105 23369 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:13.917974 23369 webserver.cc:460] Webserver started at http://127.19.202.67:42871/ using document root <none> and password file <none>
I20260810 22:39:13.918541 23369 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:13.918654 23369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:13.918882 23369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:13.920903 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "efcd36575a88406cbf5786862ce003cc"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "3c7c086d339a6fdba3b6934ec40102ef"
server_key_iv: "be413dff2becbb7b748c8c3379c815df"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.921453 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "efcd36575a88406cbf5786862ce003cc"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "3c7c086d339a6fdba3b6934ec40102ef"
server_key_iv: "be413dff2becbb7b748c8c3379c815df"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.925086 23369 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:13.927488 23384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.928627 23369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:13.928730 23369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "efcd36575a88406cbf5786862ce003cc"
format_stamp: "Formatted at 2026-08-10 22:39:13 on dist-test-slave-1x4g"
server_key: "3c7c086d339a6fdba3b6934ec40102ef"
server_key_iv: "be413dff2becbb7b748c8c3379c815df"
server_key_version: "encryptionkey@0"
I20260810 22:39:13.928862 23369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:13.949137 23369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:13.951785 23369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:13.951949 23369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:13.952457 23369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:13.953226 23369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:13.953272 23369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.953310 23369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:13.953326 23369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:13.961962 23369 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:41299
I20260810 22:39:13.962003 23507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:41299 every 8 connection(s)
I20260810 22:39:13.962811 23369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:13.968297 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 23369
I20260810 22:39:13.968423 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:13.968700 20265 external_mini_cluster.cc:1428] Setting key 1656224719b045f1899cb964ee2b28c5
Aug 10 22:39:13 dist-test-slave-1x4g krb5kdc[22952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401553, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:13.976090 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:13.964613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:36655 (local address 127.19.202.126:42395)
0810 22:39:13.964797 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:13.964802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:13.965604 (+ 802us) server_negotiation.cc:407] Connection header received
0810 22:39:13.966796 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:13.966798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:13.966844 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:13.966917 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:13.968056 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.968868 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.970266 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.970556 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.970658 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:13.973147 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:13.973150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:13.973161 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:13.973189 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:13.974422 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.974859 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.974861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.974863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.974931 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:13.975223 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:13.975225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:13.975226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:13.975382 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:13.975464 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:13.975889 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 22:39:13.975977 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:39:13.976905 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:13.965007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:42395 (local address 127.19.202.67:36655)
0810 22:39:13.965427 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:13.965467 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:13.966613 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:13.967074 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:13.967080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:13.967436 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:13.967899 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:13.967911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.968987 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.968989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:13.970152 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:13.970175 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:13.970700 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:13.970704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:13.970861 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:13.971436 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:13.971451 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:13.972982 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:13.974525 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:13.974530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:13.974540 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:13.974776 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:13.975015 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:13.975017 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:13.975018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:13.975146 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:13.975486 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:13.975491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:13.975696 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 22:39:13.975844 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 22:39:13.976068 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":99,"threads_started":1}
I20260810 22:39:13.977996 23508 heartbeater.cc:347] Connected to a master server at 127.19.202.126:42395
I20260810 22:39:13.978199 23508 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:13.978608 23508 heartbeater.cc:511] Master 127.19.202.126:42395 requested a full tablet report, sending...
I20260810 22:39:13.979516 23004 ts_manager.cc:194] Registered new tserver with Master: efcd36575a88406cbf5786862ce003cc (127.19.202.67:41299)
I20260810 22:39:13.980046 23004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:36655
I20260810 22:39:13.982357 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 22:39:13.996747 23044 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:13.995258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.19.202.126:42395)
0810 22:39:13.995390 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:13.995395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:13.995621 (+ 226us) server_negotiation.cc:407] Connection header received
0810 22:39:13.995786 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:13.995789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:13.995835 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:13.995916 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:13.996650 (+ 734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34270: BlockingRecv error: recv got EOF from 127.0.0.1:34270 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:13.997802 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 23069
I20260810 22:39:14.007052 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 23216
I20260810 22:39:14.015302 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 23369
I20260810 22:39:14.026247 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 22968
2026-08-10T22:39:14Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2162 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:14 dist-test-slave-1x4g krb5kdc[23520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:14 dist-test-slave-1x4g krb5kdc[23520](info): set up 2 sockets
Aug 10 22:39:14 dist-test-slave-1x4g krb5kdc[23520](info): commencing operation
krb5kdc: starting...
W20260810 22:39:15.095180 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:15 dist-test-slave-1x4g krb5kdc[23520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:15.211994 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:45643
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:45643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:15.318048 23535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:15.318305 23535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:15.318372 23535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:15.322328 23535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:15.322410 23535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:15.322463 23535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:15.322500 23535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:15.322536 23535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:15.327491 23535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:45643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:45643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:15.328398 23535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:15.329380 23535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:15.337682 23541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:15.337805 23540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:15.337805 23543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:15.337991 23535 server_base.cc:1034] running on GCE node
I20260810 22:39:15.338601 23535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:15.339602 23535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:15.340824 23535 hybrid_clock.cc:648] HybridClock initialized: now 1786401555340807 us; error 30 us; skew 500 ppm
Aug 10 22:39:15 dist-test-slave-1x4g krb5kdc[23520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401555, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:15.346987 23535 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:15.347842 23535 webserver.cc:460] Webserver started at http://127.19.202.126:42103/ using document root <none> and password file <none>
I20260810 22:39:15.348367 23535 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:15.348462 23535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:15.348678 23535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:15.350636 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "cb962aa222924a95a1ed5b26114b0469"
format_stamp: "Formatted at 2026-08-10 22:39:15 on dist-test-slave-1x4g"
server_key: "61570cac7ef171f3c3a396038cbbb998"
server_key_iv: "15b056adad7ada1b0c86de9e9888577e"
server_key_version: "encryptionkey@0"
I20260810 22:39:15.351189 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "cb962aa222924a95a1ed5b26114b0469"
format_stamp: "Formatted at 2026-08-10 22:39:15 on dist-test-slave-1x4g"
server_key: "61570cac7ef171f3c3a396038cbbb998"
server_key_iv: "15b056adad7ada1b0c86de9e9888577e"
server_key_version: "encryptionkey@0"
I20260810 22:39:15.354806 23535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:39:15.357074 23550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:15.358191 23535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:15.358322 23535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "cb962aa222924a95a1ed5b26114b0469"
format_stamp: "Formatted at 2026-08-10 22:39:15 on dist-test-slave-1x4g"
server_key: "61570cac7ef171f3c3a396038cbbb998"
server_key_iv: "15b056adad7ada1b0c86de9e9888577e"
server_key_version: "encryptionkey@0"
I20260810 22:39:15.358455 23535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:15.387540 23535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:15.390321 23535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:15.390532 23535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:15.397711 23535 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:45643
I20260810 22:39:15.398097 23607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:45643 every 8 connection(s)
I20260810 22:39:15.398818 23535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:15.401492 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:15.406971 23608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469: Bootstrap starting.
I20260810 22:39:15.407449 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 23535
I20260810 22:39:15.407588 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:15.407861 20265 external_mini_cluster.cc:1428] Setting key 4b7d268654db5bd9e989bc29a69193b2
I20260810 22:39:15.409708 23608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:15.410480 23608 log.cc:824] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:15.412542 23608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469: No bootstrap required, opened a new log
Aug 10 22:39:15 dist-test-slave-1x4g krb5kdc[23520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:15.416970 23608 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb962aa222924a95a1ed5b26114b0469" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45643 } }
I20260810 22:39:15.417274 23608 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:15.417344 23608 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb962aa222924a95a1ed5b26114b0469, State: Initialized, Role: FOLLOWER
I20260810 22:39:15.417809 23608 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb962aa222924a95a1ed5b26114b0469" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45643 } }
I20260810 22:39:15.417944 23608 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:15.418030 23608 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:15.418124 23608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:15.418963 23608 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb962aa222924a95a1ed5b26114b0469" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45643 } }
I20260810 22:39:15.419301 23608 leader_election.cc:302] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb962aa222924a95a1ed5b26114b0469; no voters:
I20260810 22:39:15.419595 23608 leader_election.cc:288] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:15.419713 23613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:15.419945 23613 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [term 1 LEADER]: Becoming Leader. State: Replica: cb962aa222924a95a1ed5b26114b0469, State: Running, Role: LEADER
I20260810 22:39:15.420244 23613 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb962aa222924a95a1ed5b26114b0469" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45643 } }
I20260810 22:39:15.421056 23608 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:15.421926 23616 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb962aa222924a95a1ed5b26114b0469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb962aa222924a95a1ed5b26114b0469" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45643 } } }
I20260810 22:39:15.422062 23616 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:15.422428 23617 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb962aa222924a95a1ed5b26114b0469. Latest consensus state: current_term: 1 leader_uuid: "cb962aa222924a95a1ed5b26114b0469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb962aa222924a95a1ed5b26114b0469" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 45643 } } }
I20260810 22:39:15.422534 23617 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:15.422820 23624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:15.423898 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:15.409181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.19.202.126:45643)
0810 22:39:15.409608 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:15.409623 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:15.409657 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:15.410334 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:15.410351 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:15.410720 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:15.411090 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:15.411978 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:15.413008 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:15.414195 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:15.414624 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:15.414735 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:15.417684 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:15.417690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:15.417707 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:15.417741 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:15.419658 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:15.420320 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:15.420326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:15.420332 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:15.420486 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:15.420909 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:15.420912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:15.420921 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:15.421464 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:15.421675 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:15.422591 (+ 916us) server_negotiation.cc:299] Negotiation successful
0810 22:39:15.422936 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":133,"threads_started":1}
I20260810 22:39:15.426203 23624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:15.427074 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:39:15.432293 23624 catalog_manager.cc:1347] Generated new cluster ID: ef2c38998a7542c5b54a21e8087243f5
I20260810 22:39:15.432364 23624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:15.456354 23624 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:15.457580 23624 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:15.459767 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:15.454195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41776 (local address 127.19.202.126:45643)
0810 22:39:15.454341 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:15.454346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:15.454623 (+ 277us) server_negotiation.cc:407] Connection header received
0810 22:39:15.454838 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:15.454842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:15.454886 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:15.454982 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:15.455735 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:15.456387 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:15.457511 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:15.457861 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:15.457968 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:15.458858 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:15.458862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:15.458865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:15.459272 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:15.459300 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:15.459316 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:15.459427 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:15.459535 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 22:39:15.459609 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:15.467698 23624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb962aa222924a95a1ed5b26114b0469: Generated new TSK 0
I20260810 22:39:15.468431 23624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:39:15.480458 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 23535
2026-08-10T22:39:15Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1453 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:15 dist-test-slave-1x4g krb5kdc[23643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:15 dist-test-slave-1x4g krb5kdc[23643](info): set up 2 sockets
Aug 10 22:39:15 dist-test-slave-1x4g krb5kdc[23643](info): commencing operation
krb5kdc: starting...
W20260810 22:39:16.547551 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:16 dist-test-slave-1x4g krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:16.668560 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:43825
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:43825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:16.779930 23657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:16.780174 23657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:16.780213 23657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:16.784195 23657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:16.784242 23657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:16.784278 23657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:16.784294 23657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:16.784309 23657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:16.789438 23657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:43825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:43825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:16.790424 23657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:16.791424 23657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:16.799690 23664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:16.799708 23666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:16.799710 23663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:16.799940 23657 server_base.cc:1034] running on GCE node
I20260810 22:39:16.800630 23657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:16.801697 23657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:16.802882 23657 hybrid_clock.cc:648] HybridClock initialized: now 1786401556802870 us; error 24 us; skew 500 ppm
Aug 10 22:39:16 dist-test-slave-1x4g krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401556, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:16.805576 23657 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:16.806457 23657 webserver.cc:460] Webserver started at http://127.19.202.126:43359/ using document root <none> and password file <none>
I20260810 22:39:16.806967 23657 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:16.807024 23657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:16.807186 23657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:16.809295 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "fdaf5ec89c4d419bb925052e22882de8"
format_stamp: "Formatted at 2026-08-10 22:39:16 on dist-test-slave-1x4g"
server_key: "e3aa520c8d2b507cbe4db3ce2945e21f"
server_key_iv: "627f58e2fba9e1c3c98b6f3ca6293264"
server_key_version: "encryptionkey@0"
I20260810 22:39:16.809801 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "fdaf5ec89c4d419bb925052e22882de8"
format_stamp: "Formatted at 2026-08-10 22:39:16 on dist-test-slave-1x4g"
server_key: "e3aa520c8d2b507cbe4db3ce2945e21f"
server_key_iv: "627f58e2fba9e1c3c98b6f3ca6293264"
server_key_version: "encryptionkey@0"
I20260810 22:39:16.813341 23657 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:16.815642 23673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:16.816937 23657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:16.817032 23657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "fdaf5ec89c4d419bb925052e22882de8"
format_stamp: "Formatted at 2026-08-10 22:39:16 on dist-test-slave-1x4g"
server_key: "e3aa520c8d2b507cbe4db3ce2945e21f"
server_key_iv: "627f58e2fba9e1c3c98b6f3ca6293264"
server_key_version: "encryptionkey@0"
I20260810 22:39:16.817114 23657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:16.830793 23657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:16.834705 23657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:16.834921 23657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:16.841948 23657 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:43825
I20260810 22:39:16.842228 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:43825 every 8 connection(s)
I20260810 22:39:16.842882 23657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:16.844746 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 23657
I20260810 22:39:16.844905 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:16.845232 20265 external_mini_cluster.cc:1428] Setting key c9807826a7017a56946799e4036fc835
I20260810 22:39:16.845927 23727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:16.852473 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8: Bootstrap starting.
Aug 10 22:39:16 dist-test-slave-1x4g krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:16.855062 23727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:16.855793 23727 log.cc:824] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:16.857779 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8: No bootstrap required, opened a new log
I20260810 22:39:16.859534 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:16.846525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.19.202.126:43825)
0810 22:39:16.846906 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:16.846922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:16.846955 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:39:16.847562 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:16.847577 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:16.847893 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:16.848172 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:16.849169 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:16.850183 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:16.851370 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:16.851802 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:16.851916 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:16.854697 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:16.854702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:16.854715 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:16.854740 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:16.856754 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:16.857244 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:16.857249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:16.857254 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:16.857362 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:16.857644 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:16.857647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:16.857649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:16.858111 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:16.858327 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:16.858614 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 22:39:16.858883 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":124,"threads_started":1}
I20260810 22:39:16.861639 23727 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf5ec89c4d419bb925052e22882de8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43825 } }
I20260810 22:39:16.861908 23727 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:16.861990 23727 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdaf5ec89c4d419bb925052e22882de8, State: Initialized, Role: FOLLOWER
I20260810 22:39:16.862372 23727 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf5ec89c4d419bb925052e22882de8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43825 } }
I20260810 22:39:16.862521 23727 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:16.862607 23727 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:16.862701 23727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:16.863581 23727 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf5ec89c4d419bb925052e22882de8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43825 } }
I20260810 22:39:16.863912 23727 leader_election.cc:302] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdaf5ec89c4d419bb925052e22882de8; no voters:
I20260810 22:39:16.864219 23727 leader_election.cc:288] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:16.864398 23733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:16.864694 23733 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [term 1 LEADER]: Becoming Leader. State: Replica: fdaf5ec89c4d419bb925052e22882de8, State: Running, Role: LEADER
I20260810 22:39:16.865022 23733 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf5ec89c4d419bb925052e22882de8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43825 } }
I20260810 22:39:16.865509 23727 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:16.866854 23734 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdaf5ec89c4d419bb925052e22882de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf5ec89c4d419bb925052e22882de8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43825 } } }
I20260810 22:39:16.866868 23735 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdaf5ec89c4d419bb925052e22882de8. Latest consensus state: current_term: 1 leader_uuid: "fdaf5ec89c4d419bb925052e22882de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf5ec89c4d419bb925052e22882de8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43825 } } }
I20260810 22:39:16.867012 23734 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:16.867024 23735 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:16.867442 23743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:16.870419 23743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:16.875891 23743 catalog_manager.cc:1347] Generated new cluster ID: 0ae22301d1e742ea832b3d91dea3999b
I20260810 22:39:16.875967 23743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:16.889211 23743 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:16.890089 23743 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:16.901525 23743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdaf5ec89c4d419bb925052e22882de8: Generated new TSK 0
I20260810 22:39:16.902326 23743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:16.933597 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43825
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:17.043042 23756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:17.043269 23756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:17.043308 23756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:17.046947 23756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:17.046994 23756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:17.047045 23756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:17.051815 23756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:17.052662 23756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:17.053627 23756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:17.062633 23763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:17.062709 23756 server_base.cc:1034] running on GCE node
W20260810 22:39:17.062695 23764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:17.062649 23766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:17.063333 23756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:17.063926 23756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:17.065099 23756 hybrid_clock.cc:648] HybridClock initialized: now 1786401557065092 us; error 29 us; skew 500 ppm
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:17.067968 23756 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:17.068745 23756 webserver.cc:460] Webserver started at http://127.19.202.65:41545/ using document root <none> and password file <none>
I20260810 22:39:17.069329 23756 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:17.069397 23756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:17.069638 23756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:17.071746 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "bdc11c1484454ff48a311b2594ba380d"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "5fc7fedc1fa1900eaade8caebf762873"
server_key_iv: "5f1dbe00af02b19480bbcd222462e4dd"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.072291 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "bdc11c1484454ff48a311b2594ba380d"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "5fc7fedc1fa1900eaade8caebf762873"
server_key_iv: "5f1dbe00af02b19480bbcd222462e4dd"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.076014 23756 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260810 22:39:17.078421 23773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.079535 23756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:17.079679 23756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "bdc11c1484454ff48a311b2594ba380d"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "5fc7fedc1fa1900eaade8caebf762873"
server_key_iv: "5f1dbe00af02b19480bbcd222462e4dd"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.079787 23756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:17.091426 23756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:17.094599 23756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:17.094796 23756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:17.095350 23756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:17.096161 23756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:17.096230 23756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.096298 23756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:17.096340 23756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.105386 23756 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:40587
I20260810 22:39:17.105425 23893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:40587 every 8 connection(s)
I20260810 22:39:17.106240 23756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:17.109827 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 23756
I20260810 22:39:17.109958 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:17.110258 20265 external_mini_cluster.cc:1428] Setting key 75edd4f6358bba2480f4a684955c0259
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:39:17.121445 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:17.107982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:52379 (local address 127.19.202.126:43825)
0810 22:39:17.108124 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:17.108127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:17.108993 (+ 866us) server_negotiation.cc:407] Connection header received
0810 22:39:17.110522 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:17.110526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:17.110582 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:17.110675 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:17.112092 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.112610 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.114225 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.114614 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.114726 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:17.117337 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:17.117343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:17.117348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:17.117393 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:17.119139 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.119641 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.119647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.119651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.119749 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.120073 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.120077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.120078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.120313 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:17.120431 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:17.121071 (+ 640us) server_negotiation.cc:299] Negotiation successful
0810 22:39:17.121200 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 22:39:17.122164 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:17.108346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:43825 (local address 127.19.202.65:52379)
0810 22:39:17.108854 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:17.108899 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:17.110277 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:17.110908 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:17.110917 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:17.111287 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:17.111932 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:17.111953 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.112749 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.112756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:17.114106 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:17.114116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.114760 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.114762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:17.114889 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:17.115467 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:17.115483 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:17.117161 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:17.119263 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:17.119268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:17.119278 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:17.119521 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:17.119845 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:17.119847 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:17.119849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:17.119993 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:17.120445 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:17.120449 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:17.120657 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 22:39:17.120894 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 22:39:17.121212 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":123,"threads_started":1}
I20260810 22:39:17.123359 23894 heartbeater.cc:347] Connected to a master server at 127.19.202.126:43825
I20260810 22:39:17.123605 23894 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:39:17.124192 23894 heartbeater.cc:511] Master 127.19.202.126:43825 requested a full tablet report, sending...
I20260810 22:39:17.125885 23690 ts_manager.cc:194] Registered new tserver with Master: bdc11c1484454ff48a311b2594ba380d (127.19.202.65:40587)
I20260810 22:39:17.127250 23690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:52379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:17.137806 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43825
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:17.244135 23899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:17.244366 23899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:17.244405 23899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:17.248145 23899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:17.248208 23899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:17.248260 23899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:17.253118 23899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:17.254200 23899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:17.255183 23899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:17.264599 23908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:17.264636 23905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:17.264657 23906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:17.264868 23899 server_base.cc:1034] running on GCE node
I20260810 22:39:17.265307 23899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:17.265890 23899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:17.267040 23899 hybrid_clock.cc:648] HybridClock initialized: now 1786401557267012 us; error 41 us; skew 500 ppm
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:17.269660 23899 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:17.270387 23899 webserver.cc:460] Webserver started at http://127.19.202.66:40603/ using document root <none> and password file <none>
I20260810 22:39:17.270954 23899 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:17.271014 23899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:17.271238 23899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:17.273197 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "00e22e47bed5495082318e7f7db3affe"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "400724172fd91e7e19b28f91e4dcbbec"
server_key_iv: "c713bb162a3aae16c5be48cf8f5492aa"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.273800 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "00e22e47bed5495082318e7f7db3affe"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "400724172fd91e7e19b28f91e4dcbbec"
server_key_iv: "c713bb162a3aae16c5be48cf8f5492aa"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.277292 23899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:39:17.279534 23915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.280643 23899 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:17.280766 23899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "00e22e47bed5495082318e7f7db3affe"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "400724172fd91e7e19b28f91e4dcbbec"
server_key_iv: "c713bb162a3aae16c5be48cf8f5492aa"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.280900 23899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:17.296463 23899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:17.299075 23899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:17.299265 23899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:17.299809 23899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:17.300583 23899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:17.300628 23899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.300695 23899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:17.300738 23899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.309264 23899 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:40027
I20260810 22:39:17.309288 24038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:40027 every 8 connection(s)
I20260810 22:39:17.310107 23899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:17.313995 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 23899
I20260810 22:39:17.314143 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:17.314427 20265 external_mini_cluster.cc:1428] Setting key 6a2d0e3d05f334543398a5bbcef691c6
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:17.324190 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:17.311823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:55055 (local address 127.19.202.126:43825)
0810 22:39:17.311963 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:17.311968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:17.312769 (+ 801us) server_negotiation.cc:407] Connection header received
0810 22:39:17.314138 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:17.314142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:17.314199 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:17.314309 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:17.315606 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.316183 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.317398 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.317633 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.317744 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:17.320233 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:17.320236 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:17.320238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:17.320259 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:17.322089 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.322584 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.322590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.322593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.322694 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.323050 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.323056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.323058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.323296 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:17.323406 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:17.323845 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 22:39:17.323987 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 22:39:17.324911 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:17.312158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:43825 (local address 127.19.202.66:55055)
0810 22:39:17.312629 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:17.312669 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:17.313938 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:17.314482 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:17.314488 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:17.314852 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:17.315432 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:17.315446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.316320 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.316324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:17.317272 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:17.317281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.317779 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.317784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:17.317962 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:17.318520 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:17.318537 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:17.320070 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:17.322207 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:17.322212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:17.322222 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:17.322483 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:17.322788 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:17.322791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:17.322792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:17.322949 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:17.323400 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:17.323404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:17.323608 (+ 204us) client_negotiation.cc:769] Sending connection context
0810 22:39:17.323754 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 22:39:17.324001 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:39:17.326058 24039 heartbeater.cc:347] Connected to a master server at 127.19.202.126:43825
I20260810 22:39:17.326309 24039 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:17.326779 24039 heartbeater.cc:511] Master 127.19.202.126:43825 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
I20260810 22:39:17.327759 23690 ts_manager.cc:194] Registered new tserver with Master: 00e22e47bed5495082318e7f7db3affe (127.19.202.66:40027)
I20260810 22:39:17.328367 23690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:55055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:17.341375 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43825
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:17.451283 24044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:17.451496 24044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:17.451534 24044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:17.455363 24044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:17.455415 24044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:17.455466 24044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:17.460397 24044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:17.461494 24044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:17.462402 24044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:17.471957 24051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:17.471983 24053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:17.472074 24050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:17.472177 24044 server_base.cc:1034] running on GCE node
I20260810 22:39:17.472602 24044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:17.473222 24044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:17.474385 24044 hybrid_clock.cc:648] HybridClock initialized: now 1786401557474372 us; error 27 us; skew 500 ppm
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:17.477064 24044 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:17.477871 24044 webserver.cc:460] Webserver started at http://127.19.202.67:38925/ using document root <none> and password file <none>
I20260810 22:39:17.478399 24044 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:17.478526 24044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:17.478744 24044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:17.480654 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "4bc4165514414638b8e747c57e461ffa"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "11911b792fce8a1f2d28d97d7995c1fe"
server_key_iv: "411a797d6faa7b5ed64148f7bbfe17a3"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.481236 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "4bc4165514414638b8e747c57e461ffa"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "11911b792fce8a1f2d28d97d7995c1fe"
server_key_iv: "411a797d6faa7b5ed64148f7bbfe17a3"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.485004 24044 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:17.487382 24060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.488498 24044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:17.488598 24044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "4bc4165514414638b8e747c57e461ffa"
format_stamp: "Formatted at 2026-08-10 22:39:17 on dist-test-slave-1x4g"
server_key: "11911b792fce8a1f2d28d97d7995c1fe"
server_key_iv: "411a797d6faa7b5ed64148f7bbfe17a3"
server_key_version: "encryptionkey@0"
I20260810 22:39:17.488673 24044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:17.502748 24044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:17.505386 24044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:17.505525 24044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:17.506008 24044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:17.506717 24044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:17.506760 24044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.506795 24044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:17.506810 24044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:17.515013 24044 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:35537
I20260810 22:39:17.515090 24188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:35537 every 8 connection(s)
I20260810 22:39:17.515838 24044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:17.517367 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 24044
I20260810 22:39:17.517493 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:17.517784 20265 external_mini_cluster.cc:1428] Setting key 3bbb315305e4a0350702f35753bfebd4
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:17.528380 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:17.517559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:51901 (local address 127.19.202.126:43825)
0810 22:39:17.517716 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:17.517722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:17.518708 (+ 986us) server_negotiation.cc:407] Connection header received
0810 22:39:17.520268 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:17.520270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:17.520313 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:17.520364 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:17.521472 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.521989 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.523059 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.523264 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.523338 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:17.525506 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:17.525509 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:17.525510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:17.525530 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:17.526858 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.527249 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.527252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.527253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.527321 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.527583 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.527585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.527586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.527736 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:17.527804 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:17.528176 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 22:39:17.528261 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 22:39:17.529188 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:17.517996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:43825 (local address 127.19.202.67:51901)
0810 22:39:17.518557 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:17.518594 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:17.520071 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:17.520487 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:17.520493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:17.520854 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:17.521337 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:17.521356 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.522108 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.522110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:17.522955 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:17.522964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.523382 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.523385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:17.523521 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:17.524025 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:17.524040 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:17.525359 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:17.526947 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:17.526951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:17.526960 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:17.527168 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:17.527391 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:17.527393 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:17.527394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:17.527511 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:17.527814 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:17.527818 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:17.527999 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 22:39:17.528129 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 22:39:17.528330 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":164,"threads_started":1}
I20260810 22:39:17.530218 24189 heartbeater.cc:347] Connected to a master server at 127.19.202.126:43825
I20260810 22:39:17.530441 24189 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:17.530942 24189 heartbeater.cc:511] Master 127.19.202.126:43825 requested a full tablet report, sending...
I20260810 22:39:17.531855 23690 ts_manager.cc:194] Registered new tserver with Master: 4bc4165514414638b8e747c57e461ffa (127.19.202.67:35537)
I20260810 22:39:17.532428 23690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:51901
I20260810 22:39:17.542963 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401557, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:17.567937 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:17.559798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51888 (local address 127.19.202.126:43825)
0810 22:39:17.559925 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:17.559928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:17.560089 (+ 161us) server_negotiation.cc:407] Connection header received
0810 22:39:17.560181 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:17.560183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:17.560221 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:17.560286 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:17.561007 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.561498 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.562247 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:17.562440 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:17.562509 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:17.564838 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:17.564842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:17.564849 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:17.564871 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:17.566310 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.566710 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.566712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.566714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.566831 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:17.567114 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:17.567117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:17.567118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:17.567326 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:17.567462 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:17.567668 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 22:39:17.567757 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 22:39:17.568333 23690 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51888
I20260810 22:39:17.569166 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 23756
I20260810 22:39:17.577843 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 23899
I20260810 22:39:17.588189 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 24044
I20260810 22:39:17.598438 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 23657
2026-08-10T22:39:17Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2117 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[24201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[24201](info): set up 2 sockets
Aug 10 22:39:17 dist-test-slave-1x4g krb5kdc[24201](info): commencing operation
krb5kdc: starting...
W20260810 22:39:18.659631 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:18 dist-test-slave-1x4g krb5kdc[24201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:18.775171 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39003
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:39003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:18.888271 24215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:18.888540 24215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:18.888608 24215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:18.892462 24215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:18.892541 24215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:18.892578 24215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:18.892633 24215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:18.892673 24215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:18.897778 24215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:39003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:18.898707 24215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:18.899646 24215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:18.908004 24220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:18.908013 24221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:18.908058 24215 server_base.cc:1034] running on GCE node
W20260810 22:39:18.908077 24223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:18.908741 24215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:18.909993 24215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:18.911175 24215 hybrid_clock.cc:648] HybridClock initialized: now 1786401558911156 us; error 30 us; skew 500 ppm
Aug 10 22:39:18 dist-test-slave-1x4g krb5kdc[24201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401558, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:18.913995 24215 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:18.914925 24215 webserver.cc:460] Webserver started at http://127.19.202.126:43609/ using document root <none> and password file <none>
I20260810 22:39:18.915472 24215 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:18.915562 24215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:18.915793 24215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:18.917853 24215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "41813373a6ac47a7906ea95ebeba52b2"
format_stamp: "Formatted at 2026-08-10 22:39:18 on dist-test-slave-1x4g"
server_key: "bee52bf465e050aeaf0242f19d757f55"
server_key_iv: "0e62205302f1be089fb245a8bf090ea7"
server_key_version: "encryptionkey@0"
I20260810 22:39:18.918422 24215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "41813373a6ac47a7906ea95ebeba52b2"
format_stamp: "Formatted at 2026-08-10 22:39:18 on dist-test-slave-1x4g"
server_key: "bee52bf465e050aeaf0242f19d757f55"
server_key_iv: "0e62205302f1be089fb245a8bf090ea7"
server_key_version: "encryptionkey@0"
I20260810 22:39:18.922123 24215 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:18.924474 24232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:18.925648 24215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:18.925789 24215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "41813373a6ac47a7906ea95ebeba52b2"
format_stamp: "Formatted at 2026-08-10 22:39:18 on dist-test-slave-1x4g"
server_key: "bee52bf465e050aeaf0242f19d757f55"
server_key_iv: "0e62205302f1be089fb245a8bf090ea7"
server_key_version: "encryptionkey@0"
I20260810 22:39:18.925907 24215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:18.943887 24215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:18.947006 24215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:18.947203 24215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:18.954178 24215 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:39003
I20260810 22:39:18.954195 24292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:39003 every 8 connection(s)
I20260810 22:39:18.955116 24215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:18.957772 24293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:18.960716 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 24215
I20260810 22:39:18.960897 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:18.961207 20265 external_mini_cluster.cc:1428] Setting key 94cf01de4fca7a84852868dbb75f557f
I20260810 22:39:18.963145 24293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2: Bootstrap starting.
I20260810 22:39:18.965659 24293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:18.966470 24293 log.cc:824] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:18.968407 24293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2: No bootstrap required, opened a new log
Aug 10 22:39:18 dist-test-slave-1x4g krb5kdc[24201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:18.971714 24293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41813373a6ac47a7906ea95ebeba52b2" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39003 } }
I20260810 22:39:18.972023 24293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:18.972105 24293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41813373a6ac47a7906ea95ebeba52b2, State: Initialized, Role: FOLLOWER
I20260810 22:39:18.972563 24293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41813373a6ac47a7906ea95ebeba52b2" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39003 } }
I20260810 22:39:18.972709 24293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:18.972811 24293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:18.972941 24293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:18.974051 24293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41813373a6ac47a7906ea95ebeba52b2" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39003 } }
I20260810 22:39:18.974447 24293 leader_election.cc:302] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41813373a6ac47a7906ea95ebeba52b2; no voters:
I20260810 22:39:18.974864 24293 leader_election.cc:288] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:18.974951 24299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:18.975160 24299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [term 1 LEADER]: Becoming Leader. State: Replica: 41813373a6ac47a7906ea95ebeba52b2, State: Running, Role: LEADER
I20260810 22:39:18.975489 24299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41813373a6ac47a7906ea95ebeba52b2" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39003 } }
I20260810 22:39:18.975883 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:18.962522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52332 (local address 127.19.202.126:39003)
0810 22:39:18.962891 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:18.962912 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:18.962945 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:39:18.963622 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:18.963640 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:18.963936 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:18.964203 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:18.965203 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:18.966246 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:18.967255 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:18.967710 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:18.967820 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:18.970446 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:18.970451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:18.970462 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:18.970485 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:18.972660 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:18.973210 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:18.973215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:18.973219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:18.973344 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:18.973643 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:18.973645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:18.973646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:18.974072 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:18.974297 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:18.974608 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 22:39:18.975009 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
I20260810 22:39:18.976079 24293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:18.976595 24299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41813373a6ac47a7906ea95ebeba52b2. Latest consensus state: current_term: 1 leader_uuid: "41813373a6ac47a7906ea95ebeba52b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41813373a6ac47a7906ea95ebeba52b2" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39003 } } }
I20260810 22:39:18.976722 24299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:18.977599 24300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41813373a6ac47a7906ea95ebeba52b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41813373a6ac47a7906ea95ebeba52b2" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39003 } } }
I20260810 22:39:18.977702 24300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:18.978096 24308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:18.980661 24308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:18.985625 24308 catalog_manager.cc:1347] Generated new cluster ID: e8c42504279f4bd38ff92ec1d666dc6b
I20260810 22:39:18.985689 24308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:19.011758 24308 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:19.012648 24308 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:19.016232 24308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41813373a6ac47a7906ea95ebeba52b2: Generated new TSK 0
I20260810 22:39:19.016887 24308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:19.051870 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39003
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:19.165593 24325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:19.165791 24325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:19.165827 24325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:19.169488 24325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:19.169533 24325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:19.169584 24325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:19.174966 24325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:19.175879 24325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:19.176908 24325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:19.186136 24333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:19.186154 24330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:19.186147 24331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:19.186303 24325 server_base.cc:1034] running on GCE node
I20260810 22:39:19.186733 24325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:19.187259 24325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:19.188426 24325 hybrid_clock.cc:648] HybridClock initialized: now 1786401559188401 us; error 42 us; skew 500 ppm
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:19.190972 24325 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:19.191788 24325 webserver.cc:460] Webserver started at http://127.19.202.65:36107/ using document root <none> and password file <none>
I20260810 22:39:19.192272 24325 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:19.192328 24325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:19.192513 24325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:19.194494 24325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "f94e24a3e1704b4a98f116695587fdf2"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "43144525882b39929ecd22030b1cb975"
server_key_iv: "a9c216316d07cb357cd7d1a056c23476"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.194984 24325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "f94e24a3e1704b4a98f116695587fdf2"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "43144525882b39929ecd22030b1cb975"
server_key_iv: "a9c216316d07cb357cd7d1a056c23476"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.198629 24325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:39:19.201023 24341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.202168 24325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:19.202301 24325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "f94e24a3e1704b4a98f116695587fdf2"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "43144525882b39929ecd22030b1cb975"
server_key_iv: "a9c216316d07cb357cd7d1a056c23476"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.202409 24325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:19.245451 24325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:19.248770 24325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:19.249006 24325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:19.249635 24325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:19.250491 24325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:19.250560 24325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.250628 24325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:19.250679 24325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.259243 24325 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:34091
I20260810 22:39:19.259284 24466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:34091 every 8 connection(s)
I20260810 22:39:19.260113 24325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:19.268805 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 24325
I20260810 22:39:19.268935 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:19.269222 20265 external_mini_cluster.cc:1428] Setting key 693e6f0fa20113b8b4e708292136935f
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:19.273981 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.261892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:37593 (local address 127.19.202.126:39003)
0810 22:39:19.262035 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:19.262039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:19.262825 (+ 786us) server_negotiation.cc:407] Connection header received
0810 22:39:19.264214 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:19.264217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:19.264287 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:19.264390 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:19.265588 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.266120 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.267264 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.267486 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.267586 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.270151 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:19.270159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:19.270161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:19.270184 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:19.271910 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.272374 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.272379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.272384 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.272480 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.272878 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.272882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.272883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.273089 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:19.273200 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:19.273684 (+ 484us) server_negotiation.cc:299] Negotiation successful
0810 22:39:19.273802 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:39:19.274964 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.262244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39003 (local address 127.19.202.65:37593)
0810 22:39:19.262683 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:19.262725 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:19.264005 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:19.264564 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:19.264571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:19.264968 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:19.265444 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:19.265457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.266263 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.266266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:19.267125 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:19.267133 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.267637 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.267641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:19.267795 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.268336 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:19.268350 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:19.269993 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:19.272026 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:19.272030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:19.272040 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:19.272276 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:19.272581 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:19.272583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:19.272584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:19.272714 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:19.273201 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:19.273208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:19.273445 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 22:39:19.273625 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 22:39:19.273928 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":123,"threads_started":1}
I20260810 22:39:19.276206 24467 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39003
I20260810 22:39:19.276420 24467 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:19.277068 24467 heartbeater.cc:511] Master 127.19.202.126:39003 requested a full tablet report, sending...
I20260810 22:39:19.278559 24252 ts_manager.cc:194] Registered new tserver with Master: f94e24a3e1704b4a98f116695587fdf2 (127.19.202.65:34091)
I20260810 22:39:19.279639 24252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:37593
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:19.298854 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39003
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:19.408694 24472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:19.409015 24472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:19.409093 24472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:19.412889 24472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:19.412966 24472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:19.413072 24472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:19.418253 24472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:19.419275 24472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:19.420262 24472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:19.429931 24479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:19.430040 24480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:19.429939 24482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:19.430261 24472 server_base.cc:1034] running on GCE node
I20260810 22:39:19.430792 24472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:19.431397 24472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:19.432564 24472 hybrid_clock.cc:648] HybridClock initialized: now 1786401559432537 us; error 43 us; skew 500 ppm
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:19.435431 24472 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:19.436327 24472 webserver.cc:460] Webserver started at http://127.19.202.66:37175/ using document root <none> and password file <none>
I20260810 22:39:19.436971 24472 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:19.437033 24472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:19.437266 24472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:19.439311 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "0ab94b139e694889b5ff8e4988622183"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "c7e212345422620d34f50aee9b31ab14"
server_key_iv: "4862dcdb579811ba8d917dfa05b0bcce"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.439894 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "0ab94b139e694889b5ff8e4988622183"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "c7e212345422620d34f50aee9b31ab14"
server_key_iv: "4862dcdb579811ba8d917dfa05b0bcce"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.443593 24472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 22:39:19.445979 24489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.447105 24472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:19.447228 24472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "0ab94b139e694889b5ff8e4988622183"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "c7e212345422620d34f50aee9b31ab14"
server_key_iv: "4862dcdb579811ba8d917dfa05b0bcce"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.447324 24472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:19.462900 24472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:19.466032 24472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:19.466223 24472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:19.466789 24472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:19.467571 24472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:19.467622 24472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.467685 24472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:19.467724 24472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.476974 24472 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:43593
I20260810 22:39:19.477015 24610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:43593 every 8 connection(s)
I20260810 22:39:19.478053 24472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:19.484397 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 24472
I20260810 22:39:19.484534 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:19.484848 20265 external_mini_cluster.cc:1428] Setting key edc8381e7e0848271edf20c4b11b813e
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:19.492038 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.479844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:38659 (local address 127.19.202.126:39003)
0810 22:39:19.479984 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:19.479989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:19.480754 (+ 765us) server_negotiation.cc:407] Connection header received
0810 22:39:19.482169 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:19.482173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:19.482230 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:19.482310 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:19.483418 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.483959 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.485296 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.485606 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.485720 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.488272 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:19.488276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:19.488279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:19.488306 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:19.489767 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.490261 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.490266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.490268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.490362 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.490926 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.490930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.490931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.491133 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:19.491249 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:19.491741 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 22:39:19.491873 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:39:19.493094 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.480191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39003 (local address 127.19.202.66:38659)
0810 22:39:19.480605 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:19.480657 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:19.481963 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:19.482490 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:19.482497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:19.482811 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:19.483271 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:19.483283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.484087 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.484091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:19.485173 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:19.485183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.485753 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.485757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:19.485924 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.486526 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:19.486540 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:19.488089 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:19.489898 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:19.489904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:19.489917 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:19.490159 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:19.490646 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:19.490652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:19.490654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:19.490819 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:19.491251 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:19.491258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:19.491494 (+ 236us) client_negotiation.cc:769] Sending connection context
0810 22:39:19.491684 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 22:39:19.491999 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":110,"threads_started":1}
I20260810 22:39:19.494338 24611 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39003
I20260810 22:39:19.494568 24611 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:19.495026 24611 heartbeater.cc:511] Master 127.19.202.126:39003 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:39:19.496076 24252 ts_manager.cc:194] Registered new tserver with Master: 0ab94b139e694889b5ff8e4988622183 (127.19.202.66:43593)
I20260810 22:39:19.496649 24252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:38659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:19.511508 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39003
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:19.620224 24617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:19.620491 24617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:19.620563 24617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:19.624439 24617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:19.624529 24617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:19.624616 24617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:19.629755 24617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:19.630735 24617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:19.631677 24617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:19.641139 24623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:19.641184 24624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:19.641146 24626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:19.641557 24617 server_base.cc:1034] running on GCE node
I20260810 22:39:19.641969 24617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:19.642578 24617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:19.643772 24617 hybrid_clock.cc:648] HybridClock initialized: now 1786401559643759 us; error 24 us; skew 500 ppm
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:19.646479 24617 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:19.647338 24617 webserver.cc:460] Webserver started at http://127.19.202.67:33387/ using document root <none> and password file <none>
I20260810 22:39:19.647873 24617 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:19.647961 24617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:19.648190 24617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:19.650133 24617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "6e6f672c27084c309d66a2c16bdf2c32"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "b8e3e6870e8f6e5669e4e11229b6a893"
server_key_iv: "a3fa4c477adb7e56fb2a9771ce5b7fdc"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.650743 24617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "6e6f672c27084c309d66a2c16bdf2c32"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "b8e3e6870e8f6e5669e4e11229b6a893"
server_key_iv: "a3fa4c477adb7e56fb2a9771ce5b7fdc"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.654392 24617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 22:39:19.656590 24634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.657855 24617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:19.657969 24617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "6e6f672c27084c309d66a2c16bdf2c32"
format_stamp: "Formatted at 2026-08-10 22:39:19 on dist-test-slave-1x4g"
server_key: "b8e3e6870e8f6e5669e4e11229b6a893"
server_key_iv: "a3fa4c477adb7e56fb2a9771ce5b7fdc"
server_key_version: "encryptionkey@0"
I20260810 22:39:19.658049 24617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:19.669510 24617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:19.672429 24617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:19.672564 24617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:19.673110 24617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:19.673820 24617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:19.673867 24617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.673903 24617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:19.673933 24617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:19.682401 24617 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:36621
I20260810 22:39:19.682473 24752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:36621 every 8 connection(s)
I20260810 22:39:19.683251 24617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:19.687489 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 24617
I20260810 22:39:19.687621 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:19.687880 20265 external_mini_cluster.cc:1428] Setting key 92c9ccad24a5447c43cecb38039c82b9
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:19.695813 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.684926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:49097 (local address 127.19.202.126:39003)
0810 22:39:19.685070 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:19.685073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:19.685778 (+ 705us) server_negotiation.cc:407] Connection header received
0810 22:39:19.687114 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:19.687117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:19.687175 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:19.687280 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:19.688607 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.689269 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.690367 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.690586 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.690686 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.692989 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:19.692992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:19.692994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:19.693017 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:19.694194 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.694637 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.694640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.694641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.694711 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.694993 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.694995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.694996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.695152 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:19.695223 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:19.695614 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 22:39:19.695707 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:39:19.696484 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.685243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39003 (local address 127.19.202.67:49097)
0810 22:39:19.685644 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:19.685682 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:19.686915 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:19.687453 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:19.687460 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:19.687847 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:19.688476 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:19.688491 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.689378 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.689381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:19.690253 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:19.690262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.690721 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.690725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:19.690907 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.691360 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:19.691374 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:19.692834 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:19.694305 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:19.694310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:19.694320 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:19.694554 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:19.694788 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:19.694790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:19.694791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:19.694916 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:19.695234 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:19.695238 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:19.695430 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 22:39:19.695568 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 22:39:19.695777 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":113,"threads_started":1}
I20260810 22:39:19.697669 24753 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39003
I20260810 22:39:19.697896 24753 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:19.698351 24753 heartbeater.cc:511] Master 127.19.202.126:39003 requested a full tablet report, sending...
I20260810 22:39:19.699216 24252 ts_manager.cc:194] Registered new tserver with Master: 6e6f672c27084c309d66a2c16bdf2c32 (127.19.202.67:36621)
I20260810 22:39:19.699677 24252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:49097
I20260810 22:39:19.701565 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:19.727753 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.718838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.19.202.65:34091)
0810 22:39:19.719176 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:19.719190 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:19.719212 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:39:19.719260 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:19.719265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:19.719423 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:19.719534 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:19.720142 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.720990 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.721800 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.722156 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.722235 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.724541 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:19.724549 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:19.724559 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:19.724579 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:19.725957 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.726411 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.726419 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.726426 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.726506 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.726802 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.726805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.726807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.727050 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:19.727266 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:19.727447 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 22:39:19.727589 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":147,"threads_started":1}
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:19.737845 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:19.730507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.19.202.126:39003)
0810 22:39:19.730609 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:19.730612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:19.730625 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:19.730727 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:19.730729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:19.730763 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:19.730841 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:19.731357 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.731817 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.732585 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:19.732863 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:19.732949 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:19.735088 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:19.735091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:19.735093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:19.735114 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:19.736523 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.736934 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.736936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.736938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.737008 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:19.737224 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:19.737226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:19.737227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:19.737405 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:19.737500 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:19.737653 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 22:39:19.737737 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260810 22:39:19.738217 24252 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50560
I20260810 22:39:19.738801 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 24325
I20260810 22:39:19.747181 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 24472
I20260810 22:39:19.757333 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 24617
I20260810 22:39:19.763995 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 24215
2026-08-10T22:39:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2165 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24767](info): set up 2 sockets
Aug 10 22:39:19 dist-test-slave-1x4g krb5kdc[24767](info): commencing operation
krb5kdc: starting...
W20260810 22:39:20.846735 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:20 dist-test-slave-1x4g krb5kdc[24767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:20.965658 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36249
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:36249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:21.072595 24782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:21.072865 24782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:21.072906 24782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:21.076674 24782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:21.076727 24782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:21.076746 24782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:21.076762 24782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:21.076793 24782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:21.081995 24782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:36249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:21.082872 24782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:21.083801 24782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:21.092034 24788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:21.092101 24787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:21.092141 24782 server_base.cc:1034] running on GCE node
W20260810 22:39:21.092100 24791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:21.092837 24782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:21.093891 24782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:21.095083 24782 hybrid_clock.cc:648] HybridClock initialized: now 1786401561095071 us; error 24 us; skew 500 ppm
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401561, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:21.097880 24782 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:21.098577 24782 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:21.098637 24782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:21.098793 24782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:21.100850 24782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "6198ed8d887b490b9e73792f180e9f3b"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "0a269b5e84cca25965f4a760b4bd7f0f"
server_key_iv: "0c5f64f9fabca9fc2e22c4a9f0b293ac"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.101399 24782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "6198ed8d887b490b9e73792f180e9f3b"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "0a269b5e84cca25965f4a760b4bd7f0f"
server_key_iv: "0c5f64f9fabca9fc2e22c4a9f0b293ac"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.104931 24782 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:39:21.107038 24797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.108152 24782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:21.108286 24782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "6198ed8d887b490b9e73792f180e9f3b"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "0a269b5e84cca25965f4a760b4bd7f0f"
server_key_iv: "0c5f64f9fabca9fc2e22c4a9f0b293ac"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.108384 24782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:21.119472 24782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:21.122606 24782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:21.122804 24782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:21.130090 24782 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:36249
I20260810 22:39:21.130352 24850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:36249 every 8 connection(s)
I20260810 22:39:21.130786 24782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:21.131009 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 24782
I20260810 22:39:21.131146 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:21.131462 20265 external_mini_cluster.cc:1428] Setting key 200cb174aee688734fde8d4a9e975525
I20260810 22:39:21.134109 24851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:21.140370 24851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b: Bootstrap starting.
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:21.143132 24851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:21.144049 24851 log.cc:824] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:21.146091 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.132766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.19.202.126:36249)
0810 22:39:21.133246 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.133265 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.133306 (+ 41us) server_negotiation.cc:407] Connection header received
0810 22:39:21.134190 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.134212 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.134622 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.134963 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:21.135606 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.137138 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.137978 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.138630 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.138787 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.141172 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:21.141178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:21.141191 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:21.141221 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:21.143037 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.143520 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.143525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.143531 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.143664 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.143922 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.143926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.143928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.144451 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:21.144604 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.144955 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.145228 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":116,"threads_started":1}
I20260810 22:39:21.146204 24851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b: No bootstrap required, opened a new log
I20260810 22:39:21.149801 24851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6198ed8d887b490b9e73792f180e9f3b" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36249 } }
I20260810 22:39:21.150005 24851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:21.150046 24851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6198ed8d887b490b9e73792f180e9f3b, State: Initialized, Role: FOLLOWER
I20260810 22:39:21.150426 24851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6198ed8d887b490b9e73792f180e9f3b" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36249 } }
I20260810 22:39:21.150576 24851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:21.150656 24851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:21.150758 24851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:21.151662 24851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6198ed8d887b490b9e73792f180e9f3b" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36249 } }
I20260810 22:39:21.151986 24851 leader_election.cc:302] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6198ed8d887b490b9e73792f180e9f3b; no voters:
I20260810 22:39:21.152287 24851 leader_election.cc:288] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:21.152355 24856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:21.152590 24856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [term 1 LEADER]: Becoming Leader. State: Replica: 6198ed8d887b490b9e73792f180e9f3b, State: Running, Role: LEADER
I20260810 22:39:21.152942 24856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6198ed8d887b490b9e73792f180e9f3b" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36249 } }
I20260810 22:39:21.153481 24851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:21.154546 24858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6198ed8d887b490b9e73792f180e9f3b. Latest consensus state: current_term: 1 leader_uuid: "6198ed8d887b490b9e73792f180e9f3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6198ed8d887b490b9e73792f180e9f3b" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36249 } } }
I20260810 22:39:21.154668 24858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:21.155054 24865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:21.154991 24857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6198ed8d887b490b9e73792f180e9f3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6198ed8d887b490b9e73792f180e9f3b" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36249 } } }
I20260810 22:39:21.155407 24857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:21.157960 24865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:21.163010 24865 catalog_manager.cc:1347] Generated new cluster ID: 5e1bc03184ee46ea8215ebbfeccfc6a4
I20260810 22:39:21.163069 24865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:21.170490 24865 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:21.171571 24865 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:21.177338 24865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b: Generated new TSK 0
I20260810 22:39:21.178016 24865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:21.207450 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36249
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:21.319849 24877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:21.320073 24877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:21.320163 24877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:21.324146 24877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:21.324194 24877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:21.324287 24877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:21.329072 24877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:21.330015 24877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:21.330987 24877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:21.340281 24883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:21.340276 24887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:21.340277 24884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:21.340504 24877 server_base.cc:1034] running on GCE node
I20260810 22:39:21.341073 24877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:21.341655 24877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:21.342907 24877 hybrid_clock.cc:648] HybridClock initialized: now 1786401561342892 us; error 30 us; skew 500 ppm
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401561, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:21.345916 24877 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:21.346818 24877 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:21.346969 24877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:21.347209 24877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:21.349231 24877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "7e701463aac74024825c112225164b05"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "e1abc20bd5e8b52d3e7319988d1d9107"
server_key_iv: "6bf0e06e2b1ae716fc4b4ec96c4cbb6a"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.349790 24877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "7e701463aac74024825c112225164b05"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "e1abc20bd5e8b52d3e7319988d1d9107"
server_key_iv: "6bf0e06e2b1ae716fc4b4ec96c4cbb6a"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.353277 24877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:21.355805 24894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.357003 24877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:21.357137 24877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "7e701463aac74024825c112225164b05"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "e1abc20bd5e8b52d3e7319988d1d9107"
server_key_iv: "6bf0e06e2b1ae716fc4b4ec96c4cbb6a"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.357244 24877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:21.373692 24877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:21.376818 24877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:21.377012 24877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:21.377545 24877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:21.378337 24877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:21.378407 24877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.378487 24877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:21.378531 24877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.387509 24877 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:33747
I20260810 22:39:21.387532 25018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:33747 every 8 connection(s)
I20260810 22:39:21.388068 24877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:21.393602 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 24877
I20260810 22:39:21.393749 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:21.394064 20265 external_mini_cluster.cc:1428] Setting key cb81e821ffc29f07145933b2a737bb2d
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401561, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:21.401978 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.389707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:47525 (local address 127.19.202.126:36249)
0810 22:39:21.389840 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.389843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.390624 (+ 781us) server_negotiation.cc:407] Connection header received
0810 22:39:21.391886 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.391890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.391946 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.392050 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:21.393302 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.393974 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.395150 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.395388 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.395507 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.398010 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:21.398014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:21.398016 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:21.398037 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:21.399685 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.400199 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.400205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.400210 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.400310 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.400674 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.400680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.400685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.400974 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:21.401091 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.401605 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.401754 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:39:21.402684 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.390058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36249 (local address 127.19.202.65:47525)
0810 22:39:21.390484 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:21.390524 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:21.391673 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:21.392305 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:21.392312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:21.392661 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:21.393149 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.393162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.394099 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.394105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.395019 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.395029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.395546 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.395550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.395749 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.396329 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:21.396346 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:21.397835 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:21.399808 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:21.399813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:21.399824 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:21.400095 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:21.400419 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:21.400422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:21.400423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:21.400571 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:21.401117 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:21.401123 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:21.401342 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 22:39:21.401513 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 22:39:21.401749 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":113,"threads_started":1}
I20260810 22:39:21.403950 25019 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36249
I20260810 22:39:21.404201 25019 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:21.404649 25019 heartbeater.cc:511] Master 127.19.202.126:36249 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:39:21.406153 24814 ts_manager.cc:194] Registered new tserver with Master: 7e701463aac74024825c112225164b05 (127.19.202.65:33747)
I20260810 22:39:21.407346 24814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:47525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:21.423357 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36249
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:21.535817 25024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:21.536140 25024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:21.536212 25024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:21.540443 25024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:21.540501 25024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:21.540581 25024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:21.545675 25024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:21.546607 25024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:21.547566 25024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:39:21.557106 25024 server_base.cc:1034] running on GCE node
W20260810 22:39:21.557055 25030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:21.557065 25029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:21.557057 25032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:21.557724 25024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:21.558329 25024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:21.559490 25024 hybrid_clock.cc:648] HybridClock initialized: now 1786401561559462 us; error 39 us; skew 500 ppm
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401561, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:21.562506 25024 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:21.563210 25024 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:21.563272 25024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:21.563486 25024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:21.565515 25024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "edfe198ec6f54083a841c139e4019aeb"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "563a0d0aad1332ea095807ce1deeabb1"
server_key_iv: "7f683441cff96d97720923b71090df62"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.566073 25024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "edfe198ec6f54083a841c139e4019aeb"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "563a0d0aad1332ea095807ce1deeabb1"
server_key_iv: "7f683441cff96d97720923b71090df62"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.569725 25024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:21.571985 25039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.573208 25024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:21.573329 25024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "edfe198ec6f54083a841c139e4019aeb"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "563a0d0aad1332ea095807ce1deeabb1"
server_key_iv: "7f683441cff96d97720923b71090df62"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.573426 25024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:21.603703 25024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:21.607077 25024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:21.607280 25024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:21.607857 25024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:21.608669 25024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:21.608717 25024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.608786 25024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:21.608848 25024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.617977 25024 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:44393
I20260810 22:39:21.617997 25164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:44393 every 8 connection(s)
I20260810 22:39:21.618513 25024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:21.619482 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 25024
I20260810 22:39:21.619628 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:21.619949 20265 external_mini_cluster.cc:1428] Setting key 7c102720873918c023722de437c4819b
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401561, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:39:21.633147 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.620171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:35413 (local address 127.19.202.126:36249)
0810 22:39:21.620305 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.620310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.621278 (+ 968us) server_negotiation.cc:407] Connection header received
0810 22:39:21.623029 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.623031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.623074 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.623139 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:21.624496 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.625328 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.626507 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.626967 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.627093 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.629746 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:21.629749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:21.629751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:21.629772 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:21.631227 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.631714 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.631720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.631724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.631823 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.632134 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.632136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.632137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.632300 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:21.632388 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.632936 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.633032 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
I20260810 22:39:21.634130 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.620603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36249 (local address 127.19.202.66:35413)
0810 22:39:21.621118 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:21.621173 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:21.622797 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:21.623333 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:21.623344 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:21.623844 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:21.624327 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.624340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.625456 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.625459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.626375 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.626386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.627130 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.627133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.627267 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.627812 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:21.627828 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:21.629583 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:21.631334 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:21.631339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:21.631350 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:21.631586 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:21.631907 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:21.631909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:21.631910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:21.632041 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:21.632387 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:21.632391 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:21.632651 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 22:39:21.632892 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 22:39:21.633129 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":146,"threads_started":1}
I20260810 22:39:21.635288 25165 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36249
I20260810 22:39:21.635525 25165 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:21.635955 25165 heartbeater.cc:511] Master 127.19.202.126:36249 requested a full tablet report, sending...
I20260810 22:39:21.637032 24814 ts_manager.cc:194] Registered new tserver with Master: edfe198ec6f54083a841c139e4019aeb (127.19.202.66:44393)
I20260810 22:39:21.637583 24814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:35413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:21.647290 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36249
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:21.757018 25171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:21.757262 25171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:21.757370 25171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:21.761369 25171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:21.761443 25171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:21.761524 25171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:21.766625 25171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:21.767551 25171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:21.768481 25171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:21.777472 25177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:21.777498 25178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:21.777472 25180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:21.777508 25171 server_base.cc:1034] running on GCE node
I20260810 22:39:21.778187 25171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:21.778774 25171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:21.779963 25171 hybrid_clock.cc:648] HybridClock initialized: now 1786401561779930 us; error 47 us; skew 500 ppm
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401561, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:21.782759 25171 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:21.783501 25171 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:21.783563 25171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:21.783720 25171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:21.785655 25171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "92b42a2f683143898e143c26ca917dbd"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "1d00c90a61af1b5d3f5c30f7faac34b2"
server_key_iv: "fadbabcdcccc7a8e154428cc351f9eb7"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.786160 25171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "92b42a2f683143898e143c26ca917dbd"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "1d00c90a61af1b5d3f5c30f7faac34b2"
server_key_iv: "fadbabcdcccc7a8e154428cc351f9eb7"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.789599 25171 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:21.791774 25186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.793085 25171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:21.793213 25171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "92b42a2f683143898e143c26ca917dbd"
format_stamp: "Formatted at 2026-08-10 22:39:21 on dist-test-slave-1x4g"
server_key: "1d00c90a61af1b5d3f5c30f7faac34b2"
server_key_iv: "fadbabcdcccc7a8e154428cc351f9eb7"
server_key_version: "encryptionkey@0"
I20260810 22:39:21.793334 25171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:21.821870 25171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:21.824844 25171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:21.825021 25171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:21.825590 25171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:21.826406 25171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:21.826460 25171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.826526 25171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:21.826565 25171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:21.834704 25171 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:40477
I20260810 22:39:21.834805 25309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:40477 every 8 connection(s)
I20260810 22:39:21.835281 25171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:21.843303 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 25171
I20260810 22:39:21.843480 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:21.843778 20265 external_mini_cluster.cc:1428] Setting key 372ae3204b85317715761addd0861e98
Aug 10 22:39:21 dist-test-slave-1x4g krb5kdc[24767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401561, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:21.849232 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.837026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:44901 (local address 127.19.202.126:36249)
0810 22:39:21.837161 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.837165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.838035 (+ 870us) server_negotiation.cc:407] Connection header received
0810 22:39:21.839310 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.839313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.839372 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.839478 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:21.840598 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.841220 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.842353 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.842575 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.842684 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.845609 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:21.845613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:21.845616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:21.845643 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:21.847169 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.847712 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.847737 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.847738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.847814 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.848160 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.848163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.848165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.848339 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:21.848447 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.848957 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.849080 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:39:21.850028 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.837418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36249 (local address 127.19.202.67:44901)
0810 22:39:21.837863 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:21.837935 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:21.839105 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:21.839643 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:21.839650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:21.840018 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:21.840457 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.840470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.841351 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.841354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.842229 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.842237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.842719 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.842723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.842884 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.843552 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:21.843576 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:21.845441 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:21.847295 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:21.847300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:21.847313 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:21.847603 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:21.847915 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:21.847919 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:21.847921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:21.848060 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:21.848448 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:21.848455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:21.848688 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 22:39:21.848890 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 22:39:21.849192 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
I20260810 22:39:21.851002 25310 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36249
I20260810 22:39:21.851238 25310 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:21.851663 25310 heartbeater.cc:511] Master 127.19.202.126:36249 requested a full tablet report, sending...
I20260810 22:39:21.852597 24814 ts_manager.cc:194] Registered new tserver with Master: 92b42a2f683143898e143c26ca917dbd (127.19.202.67:40477)
I20260810 22:39:21.853298 24814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:44901
I20260810 22:39:21.857618 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:39:21.866339 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.859506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.19.202.126:36249)
0810 22:39:21.859628 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.859631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.859895 (+ 264us) server_negotiation.cc:407] Connection header received
0810 22:39:21.859972 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.859974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.860010 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.860068 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:21.860755 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.861276 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.862072 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.862268 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.862344 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.863468 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:21.863471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:21.863472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:21.863493 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:21.864918 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.865284 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.865286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.865287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.865359 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:21.865582 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:21.865584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:21.865586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:21.865781 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:21.865914 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.866066 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.866149 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:21.868841 24814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:21.871658 24814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:21.886220 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.880533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:40477 (local address 127.0.0.1:38692)
0810 22:39:21.881149 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:21.881162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:21.881245 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:21.882099 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:21.882102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:21.882120 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:21.882238 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.882247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.883254 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.883256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.884776 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.884812 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.885714 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.885717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.885817 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.885855 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:39:21.885964 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 22:39:21.886077 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":186,"thread_start_us":95,"threads_started":1}
I20260810 22:39:21.886225 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.880672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.19.202.67:40477)
0810 22:39:21.881466 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.881480 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.881503 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:39:21.881822 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.881828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.881969 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.882119 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:21.882362 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.883075 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.884947 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.885585 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.885702 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.885838 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.885995 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.886077 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":128,"threads_started":1}
I20260810 22:39:21.886626 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.879721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:44393 (local address 127.0.0.1:51378)
0810 22:39:21.880243 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:21.880287 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:21.880421 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:21.881258 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:21.881264 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:21.881289 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:21.881449 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.881455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.883254 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.883257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.884758 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.884802 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.886066 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.886068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.886270 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.886350 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 22:39:21.886418 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 22:39:21.886483 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":269,"thread_start_us":162,"threads_started":1}
I20260810 22:39:21.887117 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.880566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.19.202.65:33747)
0810 22:39:21.881005 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.881011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.881032 (+ 21us) server_negotiation.cc:407] Connection header received
0810 22:39:21.881105 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.881109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.881276 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.881689 (+ 413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:21.881863 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.883044 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.885149 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.885926 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.886359 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.886516 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.886877 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.886966 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":126,"threads_started":1}
I20260810 22:39:21.887439 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.879908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:33747 (local address 127.0.0.1:34202)
0810 22:39:21.880854 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:21.880868 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:21.880957 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:21.881422 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:21.881426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:21.881440 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:21.881528 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.881537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.883210 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.883213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.885025 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:21.885034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.886064 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.886068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:21.886269 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.886774 (+ 505us) client_negotiation.cc:769] Sending connection context
0810 22:39:21.886841 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 22:39:21.886918 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":643,"thread_start_us":102,"threads_started":1}
I20260810 22:39:21.888147 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:21.880056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.19.202.66:44393)
0810 22:39:21.880741 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:21.880753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:21.880776 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:39:21.880874 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:21.880880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:21.881082 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:21.881616 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:21.881855 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.883033 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.884921 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:21.885755 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:21.885863 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:21.886018 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:21.887908 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0810 22:39:21.888004 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":117,"threads_started":1}
I20260810 22:39:21.889621 25236 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9333b2f2a548ce8f755b324bea2934 (DEFAULT_TABLE table=test-table [id=6322e29bcae142329240d3044fd27beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:21.890156 24945 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9333b2f2a548ce8f755b324bea2934 (DEFAULT_TABLE table=test-table [id=6322e29bcae142329240d3044fd27beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:21.890576 25236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9333b2f2a548ce8f755b324bea2934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:21.890770 25092 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9333b2f2a548ce8f755b324bea2934 (DEFAULT_TABLE table=test-table [id=6322e29bcae142329240d3044fd27beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:21.891151 24945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9333b2f2a548ce8f755b324bea2934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:21.891717 25092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9333b2f2a548ce8f755b324bea2934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:21.896327 25330 tablet_bootstrap.cc:492] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Bootstrap starting.
I20260810 22:39:21.898185 25331 tablet_bootstrap.cc:492] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05: Bootstrap starting.
I20260810 22:39:21.898595 25330 tablet_bootstrap.cc:654] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:21.899199 25332 tablet_bootstrap.cc:492] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Bootstrap starting.
I20260810 22:39:21.899976 25330 log.cc:824] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:21.900437 25331 tablet_bootstrap.cc:654] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:21.901134 25332 tablet_bootstrap.cc:654] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:21.901685 25331 log.cc:824] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:21.902220 25330 tablet_bootstrap.cc:492] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: No bootstrap required, opened a new log
I20260810 22:39:21.902340 25332 log.cc:824] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:21.902433 25330 ts_tablet_manager.cc:1397] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:39:21.903759 25331 tablet_bootstrap.cc:492] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05: No bootstrap required, opened a new log
I20260810 22:39:21.903945 25331 ts_tablet_manager.cc:1397] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:39:21.904167 25332 tablet_bootstrap.cc:492] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: No bootstrap required, opened a new log
I20260810 22:39:21.904356 25332 ts_tablet_manager.cc:1397] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:39:21.906438 25330 raft_consensus.cc:348] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:21.906700 25330 raft_consensus.cc:374] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:21.906780 25330 raft_consensus.cc:729] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b42a2f683143898e143c26ca917dbd, State: Initialized, Role: FOLLOWER
I20260810 22:39:21.907251 25330 consensus_queue.cc:260] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:21.907392 25332 raft_consensus.cc:348] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:21.907716 25332 raft_consensus.cc:374] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:21.907585 25331 raft_consensus.cc:348] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:21.907784 25332 raft_consensus.cc:729] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edfe198ec6f54083a841c139e4019aeb, State: Initialized, Role: FOLLOWER
I20260810 22:39:21.907874 25331 raft_consensus.cc:374] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:21.907938 25331 raft_consensus.cc:729] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e701463aac74024825c112225164b05, State: Initialized, Role: FOLLOWER
I20260810 22:39:21.908245 25332 consensus_queue.cc:260] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:21.908376 25330 ts_tablet_manager.cc:1428] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 22:39:21.908419 25331 consensus_queue.cc:260] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:21.908748 25310 heartbeater.cc:503] Master 127.19.202.126:36249 was elected leader, sending a full tablet report...
I20260810 22:39:21.908927 25165 heartbeater.cc:503] Master 127.19.202.126:36249 was elected leader, sending a full tablet report...
I20260810 22:39:21.909263 25332 ts_tablet_manager.cc:1428] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:39:21.909338 25331 ts_tablet_manager.cc:1428] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:39:21.909672 25019 heartbeater.cc:503] Master 127.19.202.126:36249 was elected leader, sending a full tablet report...
I20260810 22:39:22.014971 25338 raft_consensus.cc:482] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:22.015292 25338 raft_consensus.cc:504] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:22.016413 25338 leader_election.cc:288] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edfe198ec6f54083a841c139e4019aeb (127.19.202.66:44393), 92b42a2f683143898e143c26ca917dbd (127.19.202.67:40477)
I20260810 22:39:22.020452 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.016797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:40237 (local address 127.19.202.67:40477)
0810 22:39:22.017064 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.017071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.017156 (+ 85us) server_negotiation.cc:407] Connection header received
0810 22:39:22.017359 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.017364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.017426 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.017485 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:22.017858 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.018325 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.019408 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.019826 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.019949 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:22.020001 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.020245 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.020325 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260810 22:39:22.020478 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.016673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:44393 (local address 127.19.202.65:42385)
0810 22:39:22.017156 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.017168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.017283 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.017577 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.017581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.017622 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:22.017726 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:22.017730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.018466 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.018469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:22.019298 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:22.019307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.019991 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.019994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:22.020116 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:22.020132 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.020252 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.020324 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":251,"spinlock_wait_cycles":896,"thread_start_us":107,"threads_started":1}
I20260810 22:39:22.020466 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.016846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:42385 (local address 127.19.202.66:44393)
0810 22:39:22.017009 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.017014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.017254 (+ 240us) server_negotiation.cc:407] Connection header received
0810 22:39:22.017403 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.017405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.017448 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.017543 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:22.017888 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.018336 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.019474 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.019847 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.019988 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:22.020039 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.020298 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.020373 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:39:22.020488 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.016673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:40477 (local address 127.19.202.65:40237)
0810 22:39:22.017056 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.017072 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.017191 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.017531 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.017535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.017551 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:22.017714 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:22.017720 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.018448 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.018453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:22.019259 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:22.019267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.019971 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.019975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:22.020070 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:22.020104 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.020195 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.020291 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":4,"spinlock_wait_cycles":29184,"thread_start_us":163,"threads_started":1}
I20260810 22:39:22.021059 25116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9333b2f2a548ce8f755b324bea2934" candidate_uuid: "7e701463aac74024825c112225164b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edfe198ec6f54083a841c139e4019aeb" is_pre_election: true
I20260810 22:39:22.021102 25259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9333b2f2a548ce8f755b324bea2934" candidate_uuid: "7e701463aac74024825c112225164b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b42a2f683143898e143c26ca917dbd" is_pre_election: true
I20260810 22:39:22.021334 25116 raft_consensus.cc:2454] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e701463aac74024825c112225164b05 in term 0.
I20260810 22:39:22.021353 25259 raft_consensus.cc:2454] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e701463aac74024825c112225164b05 in term 0.
I20260810 22:39:22.021749 24898 leader_election.cc:302] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e701463aac74024825c112225164b05, edfe198ec6f54083a841c139e4019aeb; no voters:
I20260810 22:39:22.022023 25338 raft_consensus.cc:2781] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:22.022102 25338 raft_consensus.cc:482] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:22.022131 25338 raft_consensus.cc:3037] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:22.023149 25338 raft_consensus.cc:504] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:22.023532 25338 leader_election.cc:288] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [CANDIDATE]: Term 1 election: Requested vote from peers edfe198ec6f54083a841c139e4019aeb (127.19.202.66:44393), 92b42a2f683143898e143c26ca917dbd (127.19.202.67:40477)
I20260810 22:39:22.023813 25116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9333b2f2a548ce8f755b324bea2934" candidate_uuid: "7e701463aac74024825c112225164b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edfe198ec6f54083a841c139e4019aeb"
I20260810 22:39:22.023877 25259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9333b2f2a548ce8f755b324bea2934" candidate_uuid: "7e701463aac74024825c112225164b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b42a2f683143898e143c26ca917dbd"
I20260810 22:39:22.023938 25116 raft_consensus.cc:3037] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:22.023978 25259 raft_consensus.cc:3037] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:22.025274 25116 raft_consensus.cc:2454] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e701463aac74024825c112225164b05 in term 1.
I20260810 22:39:22.025274 25259 raft_consensus.cc:2454] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e701463aac74024825c112225164b05 in term 1.
I20260810 22:39:22.025591 24895 leader_election.cc:302] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e701463aac74024825c112225164b05, 92b42a2f683143898e143c26ca917dbd; no voters:
I20260810 22:39:22.025745 25338 raft_consensus.cc:2781] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:22.025940 25338 raft_consensus.cc:686] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 1 LEADER]: Becoming Leader. State: Replica: 7e701463aac74024825c112225164b05, State: Running, Role: LEADER
I20260810 22:39:22.026264 25338 consensus_queue.cc:237] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } }
I20260810 22:39:22.029099 24813 catalog_manager.cc:5495] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e701463aac74024825c112225164b05 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "7e701463aac74024825c112225164b05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e701463aac74024825c112225164b05" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 33747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:22.040562 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.037517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.19.202.65:33747)
0810 22:39:22.037733 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.037737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.037750 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:22.037818 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.037820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.037862 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.037942 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:22.038147 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.038596 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.039362 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:22.039556 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:22.039625 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:22.039861 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:22.040209 (+ 348us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:22.040293 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.040411 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.040456 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260810 22:39:22.046005 25116 raft_consensus.cc:1261] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 1 FOLLOWER]: Refusing update from remote peer 7e701463aac74024825c112225164b05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:22.046006 25259 raft_consensus.cc:1261] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 1 FOLLOWER]: Refusing update from remote peer 7e701463aac74024825c112225164b05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:22.046620 25338 consensus_queue.cc:1035] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92b42a2f683143898e143c26ca917dbd" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:22.046787 25341 consensus_queue.cc:1035] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edfe198ec6f54083a841c139e4019aeb" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:22.056890 25344 mvcc.cc:205] Tried to move back new op lower bound from 7317100798133534720 to 7317100798061981696. Current Snapshot: MvccSnapshot[applied={T|T < 7317100798133534720 or (T in {7317100798133534720})}]
I20260810 22:39:22.063117 24813 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:22.063287 24813 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:22.065723 24813 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6198ed8d887b490b9e73792f180e9f3b: Sending DeleteTablet for 3 replicas of tablet 4f9333b2f2a548ce8f755b324bea2934
I20260810 22:39:22.066455 25092 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9333b2f2a548ce8f755b324bea2934 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:22 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:51378
I20260810 22:39:22.066574 25236 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9333b2f2a548ce8f755b324bea2934 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:22 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:38692
I20260810 22:39:22.066706 24945 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9333b2f2a548ce8f755b324bea2934 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:22 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:34202
I20260810 22:39:22.066936 25357 tablet_replica.cc:330] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: stopping tablet replica
I20260810 22:39:22.067183 25358 tablet_replica.cc:330] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05: stopping tablet replica
I20260810 22:39:22.067257 25357 raft_consensus.cc:2229] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:22.067483 25358 raft_consensus.cc:2229] T 4f9333b2f2a548ce8f755b324bea2934 P 7e701463aac74024825c112225164b05 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:22.067580 25357 raft_consensus.cc:2258] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:22.067689 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 24877
I20260810 22:39:22.069059 25357 ts_tablet_manager.cc:1905] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:22.071204 25357 ts_tablet_manager.cc:1918] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:22.071195 25356 tablet_replica.cc:330] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: stopping tablet replica
I20260810 22:39:22.071321 25357 log.cc:1196] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/4f9333b2f2a548ce8f755b324bea2934
I20260810 22:39:22.071501 25356 raft_consensus.cc:2229] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:22.071699 25357 ts_tablet_manager.cc:1939] T 4f9333b2f2a548ce8f755b324bea2934 P 92b42a2f683143898e143c26ca917dbd: Deleting consensus metadata
I20260810 22:39:22.071852 25356 raft_consensus.cc:2258] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:22.072911 24798 catalog_manager.cc:4841] TS 92b42a2f683143898e143c26ca917dbd (127.19.202.67:40477): tablet 4f9333b2f2a548ce8f755b324bea2934 (table test-table [id=6322e29bcae142329240d3044fd27beb]) successfully deleted
I20260810 22:39:22.073609 25356 ts_tablet_manager.cc:1905] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:22.077159 25356 ts_tablet_manager.cc:1918] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:22.077286 25356 log.cc:1196] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/4f9333b2f2a548ce8f755b324bea2934
I20260810 22:39:22.077654 25356 ts_tablet_manager.cc:1939] T 4f9333b2f2a548ce8f755b324bea2934 P edfe198ec6f54083a841c139e4019aeb: Deleting consensus metadata
I20260810 22:39:22.078895 24801 catalog_manager.cc:4841] TS edfe198ec6f54083a841c139e4019aeb (127.19.202.66:44393): tablet 4f9333b2f2a548ce8f755b324bea2934 (table test-table [id=6322e29bcae142329240d3044fd27beb]) successfully deleted
W20260810 22:39:22.079828 24799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:33747 (error 108)
I20260810 22:39:22.080725 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 25024
I20260810 22:39:22.082115 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.080856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:33747 (local address 127.0.0.1:34210)
0810 22:39:22.081926 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.082013 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":961}
W20260810 22:39:22.087301 24799 catalog_manager.cc:4584] TS 7e701463aac74024825c112225164b05 (127.19.202.65:33747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f9333b2f2a548ce8f755b324bea2934: Network error: Client connection negotiation failed: client connection to 127.19.202.65:33747: connect: Connection refused (error 111)
I20260810 22:39:22.092576 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 25171
I20260810 22:39:22.099676 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 24782
2026-08-10T22:39:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2335 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T22:39:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:22Z Disabled control of system clock
I20260810 22:39:22.132061 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36611
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:36611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 22:39:22.240975 25364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:22.241194 25364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:22.241226 25364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:22.245262 25364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:22.245324 25364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:22.245342 25364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:22.245359 25364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:22.245406 25364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:22.249966 25364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:36611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:22.250937 25364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:22.252269 25364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:22.260430 25370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:22.260560 25369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:22.260581 25373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:22.260859 25364 server_base.cc:1034] running on GCE node
I20260810 22:39:22.261405 25364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:22.262346 25364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:22.263512 25364 hybrid_clock.cc:648] HybridClock initialized: now 1786401562263494 us; error 29 us; skew 500 ppm
I20260810 22:39:22.265308 25364 webserver.cc:460] Webserver started at http://127.19.202.126:36359/ using document root <none> and password file <none>
I20260810 22:39:22.265857 25364 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:22.265918 25364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:22.266140 25364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:22.268131 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "7ee4b11db760489ab9bcad3b4bd44beb"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "150853962256c8f3207baa4ae867578a"
server_key_iv: "7bde05fe02bce0edc1d7d26d8e07cd9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.268668 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "7ee4b11db760489ab9bcad3b4bd44beb"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "150853962256c8f3207baa4ae867578a"
server_key_iv: "7bde05fe02bce0edc1d7d26d8e07cd9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.272121 25364 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:22.274504 25379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.275635 25364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 22:39:22.275771 25364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "7ee4b11db760489ab9bcad3b4bd44beb"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "150853962256c8f3207baa4ae867578a"
server_key_iv: "7bde05fe02bce0edc1d7d26d8e07cd9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.275878 25364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:22.280187 25364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:22.280864 25364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:22.281035 25364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:22.287586 25364 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:36611
I20260810 22:39:22.287608 25435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:36611 every 8 connection(s)
I20260810 22:39:22.288372 25364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:22.291414 25437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:22.296737 25437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb: Bootstrap starting.
I20260810 22:39:22.297480 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 25364
I20260810 22:39:22.297600 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:22.297906 20265 external_mini_cluster.cc:1428] Setting key 3f2279bc087ce2d90a518060c24d7da0
I20260810 22:39:22.299655 25437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:22.300354 25437 log.cc:824] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:22.302417 25437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb: No bootstrap required, opened a new log
I20260810 22:39:22.303364 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.299160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.19.202.126:36611)
0810 22:39:22.299893 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.299908 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.299956 (+ 48us) server_negotiation.cc:407] Connection header received
0810 22:39:22.300592 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.300611 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.300989 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.301349 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.302149 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.302153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.302165 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.302175 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.302229 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.302244 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.302362 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.302494 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.302727 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":137,"threads_started":1}
I20260810 22:39:22.306504 25437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36611 } }
I20260810 22:39:22.306723 25437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:22.306806 25437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee4b11db760489ab9bcad3b4bd44beb, State: Initialized, Role: FOLLOWER
I20260810 22:39:22.307266 25437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36611 } }
I20260810 22:39:22.307432 25437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:22.307549 25437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:22.307659 25437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:22.308470 25437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36611 } }
I20260810 22:39:22.308840 25437 leader_election.cc:302] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ee4b11db760489ab9bcad3b4bd44beb; no voters:
I20260810 22:39:22.309142 25437 leader_election.cc:288] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:22.309254 25442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:22.309487 25442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [term 1 LEADER]: Becoming Leader. State: Replica: 7ee4b11db760489ab9bcad3b4bd44beb, State: Running, Role: LEADER
I20260810 22:39:22.309854 25442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36611 } }
I20260810 22:39:22.310333 25437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:22.311658 25445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ee4b11db760489ab9bcad3b4bd44beb. Latest consensus state: current_term: 1 leader_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36611 } } }
I20260810 22:39:22.311805 25445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:22.312125 25444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee4b11db760489ab9bcad3b4bd44beb" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36611 } } }
I20260810 22:39:22.312196 25454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:22.312278 25444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:22.314822 25454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:22.319945 25454 catalog_manager.cc:1347] Generated new cluster ID: 7b5c4f9168f04417b8795f4251bb060b
I20260810 22:39:22.320016 25454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:22.328461 25454 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:22.328748 25454 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:22.333206 25454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb: Generated new TSK 0
I20260810 22:39:22.333822 25454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:39:22.340850 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36611
--builtin_ntp_servers=127.19.202.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 22:39:22.454772 25464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:22.455011 25464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:22.455047 25464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:22.459059 25464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:22.459110 25464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:22.459196 25464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:22.463806 25464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:22.464906 25464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:22.466333 25464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:22.475570 25469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:22.475646 25464 server_base.cc:1034] running on GCE node
W20260810 22:39:22.475557 25472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:22.476302 25470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:22.476737 25464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:22.477358 25464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:22.478513 25464 hybrid_clock.cc:648] HybridClock initialized: now 1786401562478482 us; error 42 us; skew 500 ppm
I20260810 22:39:22.480206 25464 webserver.cc:460] Webserver started at http://127.19.202.65:41641/ using document root <none> and password file <none>
I20260810 22:39:22.480860 25464 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:22.480929 25464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:22.481218 25464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:22.483175 25464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "b5653bb2b3b74a89a99bd3695bc02ecc"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "fc43b5ce3db00286ad55625a41e05ef1"
server_key_iv: "02e050047c4aeae34564a9a0336fe53d"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.483754 25464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "b5653bb2b3b74a89a99bd3695bc02ecc"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "fc43b5ce3db00286ad55625a41e05ef1"
server_key_iv: "02e050047c4aeae34564a9a0336fe53d"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.487437 25464 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 22:39:22.489821 25479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.490989 25464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:22.491106 25464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "b5653bb2b3b74a89a99bd3695bc02ecc"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "fc43b5ce3db00286ad55625a41e05ef1"
server_key_iv: "02e050047c4aeae34564a9a0336fe53d"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.491209 25464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:22.495643 25464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:22.496212 25464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:22.496388 25464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:22.496964 25464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:22.497838 25464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:22.497896 25464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.497963 25464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:22.497993 25464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.506237 25464 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:43215
I20260810 22:39:22.506301 25605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:43215 every 8 connection(s)
I20260810 22:39:22.507081 25464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:22.508517 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 25464
I20260810 22:39:22.508651 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:22.508988 20265 external_mini_cluster.cc:1428] Setting key d6699fe4179a28ac877f48706bca74db
I20260810 22:39:22.511260 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36611
--builtin_ntp_servers=127.19.202.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 22:39:22.513784 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.509016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:58085 (local address 127.19.202.126:36611)
0810 22:39:22.509143 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.509152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.510201 (+ 1049us) server_negotiation.cc:407] Connection header received
0810 22:39:22.511966 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.511969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.512002 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.512065 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.513294 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.513297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.513300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.513313 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.513334 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.513345 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.513447 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.513612 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.513658 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:22.514644 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.509323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36611 (local address 127.19.202.65:58085)
0810 22:39:22.510074 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.510112 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.511697 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.512219 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.512230 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.512333 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:22.512981 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:22.512984 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:22.513042 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:22.513137 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:22.513434 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:22.513444 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:22.513450 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.513634 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.513864 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":113,"threads_started":1}
I20260810 22:39:22.515873 25606 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36611
I20260810 22:39:22.516126 25606 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:22.516697 25606 heartbeater.cc:511] Master 127.19.202.126:36611 requested a full tablet report, sending...
I20260810 22:39:22.518538 25396 ts_manager.cc:194] Registered new tserver with Master: b5653bb2b3b74a89a99bd3695bc02ecc (127.19.202.65:43215)
W20260810 22:39:22.621760 25609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:22.621994 25609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:22.622030 25609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:22.625828 25609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:22.625867 25609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:22.625922 25609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:22.630074 25609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:22.630987 25609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:22.632210 25609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:22.641317 25614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:22.641397 25609 server_base.cc:1034] running on GCE node
W20260810 22:39:22.641443 25617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:22.641345 25615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:22.642041 25609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:22.642680 25609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:22.643862 25609 hybrid_clock.cc:648] HybridClock initialized: now 1786401562643826 us; error 48 us; skew 500 ppm
I20260810 22:39:22.645634 25609 webserver.cc:460] Webserver started at http://127.19.202.66:38101/ using document root <none> and password file <none>
I20260810 22:39:22.646219 25609 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:22.646309 25609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:22.646610 25609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:22.648622 25609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "1bc451e494b346469adcc6fa2870ed88"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "b34808c20883b93bdd3d628fc66b2592"
server_key_iv: "ac41222bb4a90f9d5e28ca40e8ec8e59"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.649286 25609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "1bc451e494b346469adcc6fa2870ed88"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "b34808c20883b93bdd3d628fc66b2592"
server_key_iv: "ac41222bb4a90f9d5e28ca40e8ec8e59"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.653241 25609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 22:39:22.655700 25623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.656994 25609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:22.657121 25609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "1bc451e494b346469adcc6fa2870ed88"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "b34808c20883b93bdd3d628fc66b2592"
server_key_iv: "ac41222bb4a90f9d5e28ca40e8ec8e59"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.657232 25609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:22.661759 25609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:22.662348 25609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:22.662526 25609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:22.663054 25609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:22.663972 25609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:22.664045 25609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.664112 25609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:22.664161 25609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.672847 25609 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:34473
I20260810 22:39:22.672864 25745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:34473 every 8 connection(s)
I20260810 22:39:22.673734 25609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:22.678192 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 25609
I20260810 22:39:22.678314 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:22.678627 20265 external_mini_cluster.cc:1428] Setting key 996222e822a99311f71748a5ec410fb8
I20260810 22:39:22.680269 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.675637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:32947 (local address 127.19.202.126:36611)
0810 22:39:22.675784 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.675787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.676877 (+ 1090us) server_negotiation.cc:407] Connection header received
0810 22:39:22.678205 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.678212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.678262 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.678367 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.679767 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.679770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.679773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.679784 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.679803 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.679811 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.679884 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.680068 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.680111 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:39:22.681000 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36611
--builtin_ntp_servers=127.19.202.84:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 22:39:22.681254 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.675971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36611 (local address 127.19.202.66:32947)
0810 22:39:22.676678 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.676718 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.677976 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.678534 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.678540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.678625 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:22.679309 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:22.679310 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:22.679513 (+ 203us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:22.679615 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:22.679908 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:22.679917 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:22.679923 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.680107 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.680330 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":136,"threads_started":1}
I20260810 22:39:22.682430 25746 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36611
I20260810 22:39:22.682690 25746 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:22.683106 25746 heartbeater.cc:511] Master 127.19.202.126:36611 requested a full tablet report, sending...
I20260810 22:39:22.684196 25396 ts_manager.cc:194] Registered new tserver with Master: 1bc451e494b346469adcc6fa2870ed88 (127.19.202.66:34473)
W20260810 22:39:22.796166 25752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:22.796439 25752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:22.796530 25752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:22.800457 25752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:22.800535 25752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:22.800624 25752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:22.805423 25752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:22.806497 25752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:22.807900 25752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:22.817523 25757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:22.817632 25758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:22.817564 25760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:22.817770 25752 server_base.cc:1034] running on GCE node
I20260810 22:39:22.818305 25752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:22.818928 25752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:22.820109 25752 hybrid_clock.cc:648] HybridClock initialized: now 1786401562820077 us; error 44 us; skew 500 ppm
I20260810 22:39:22.821946 25752 webserver.cc:460] Webserver started at http://127.19.202.67:46555/ using document root <none> and password file <none>
I20260810 22:39:22.822512 25752 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:22.822605 25752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:22.822873 25752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:22.825052 25752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "439bd9e7608a329e0d6427e4fad3892a"
server_key_iv: "ae262847034745925f57512046ca33b8"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.825620 25752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "439bd9e7608a329e0d6427e4fad3892a"
server_key_iv: "ae262847034745925f57512046ca33b8"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.830075 25752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 22:39:22.833516 25767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.835000 25752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:22.835115 25752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5"
format_stamp: "Formatted at 2026-08-10 22:39:22 on dist-test-slave-1x4g"
server_key: "439bd9e7608a329e0d6427e4fad3892a"
server_key_iv: "ae262847034745925f57512046ca33b8"
server_key_version: "encryptionkey@0"
I20260810 22:39:22.835246 25752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:22.840435 25752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:22.841064 25752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:22.841238 25752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:22.841766 25752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:22.842657 25752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:22.842702 25752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.842767 25752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:22.842804 25752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:22.851310 25752 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:43935
I20260810 22:39:22.851337 25885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:43935 every 8 connection(s)
I20260810 22:39:22.852118 25752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:22.858219 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.853924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:48815 (local address 127.19.202.126:36611)
0810 22:39:22.854054 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.854057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.855024 (+ 967us) server_negotiation.cc:407] Connection header received
0810 22:39:22.856341 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.856344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.856390 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.856499 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.857697 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.857700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.857703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.857717 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.857747 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.857755 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.857863 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.858029 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.858077 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:22.859010 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.854230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36611 (local address 127.19.202.67:48815)
0810 22:39:22.854863 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.854900 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.856132 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.856693 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.856700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.856829 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:22.857440 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:22.857442 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:22.857480 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:22.857546 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:22.857846 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:22.857854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:22.857860 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.858059 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.858299 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":121,"threads_started":1}
I20260810 22:39:22.860123 25886 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36611
I20260810 22:39:22.860432 25886 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:22.860553 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 25752
I20260810 22:39:22.860680 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:22.860926 25886 heartbeater.cc:511] Master 127.19.202.126:36611 requested a full tablet report, sending...
I20260810 22:39:22.861016 20265 external_mini_cluster.cc:1428] Setting key 69b1f3cd4aa018b4274e0dced0f9a300
I20260810 22:39:22.862052 25396 ts_manager.cc:194] Registered new tserver with Master: 2d76113cbd9e48e59f0a74e7efd4a7e5 (127.19.202.67:43935)
I20260810 22:39:22.863155 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:39:22.866541 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.865276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.19.202.126:36611)
0810 22:39:22.865383 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.865386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.865558 (+ 172us) server_negotiation.cc:407] Connection header received
0810 22:39:22.865650 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.865652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.865678 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.865741 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.866287 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.866290 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.866292 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.866302 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.866317 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.866324 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.866386 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.866429 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.866457 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:39:22.869277 25395 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:22.872169 25395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:22.884217 25901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.881049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:43215 (local address 127.0.0.1:34796)
0810 22:39:22.881570 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.881582 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.881755 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.882481 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.882483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.882560 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:22.883344 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:22.883346 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:22.883353 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:22.883423 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:22.883875 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:22.883886 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:22.883895 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.884006 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.884075 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":114,"threads_started":1}
I20260810 22:39:22.884346 25899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.880436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.19.202.67:43935)
0810 22:39:22.881094 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.881106 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.881397 (+ 291us) server_negotiation.cc:407] Connection header received
0810 22:39:22.881587 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.881597 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.881776 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.881893 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.883622 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.883631 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.883646 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.883664 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.883714 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.883751 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.883875 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.884067 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.884150 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":105,"threads_started":1}
I20260810 22:39:22.884596 25900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.880444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.19.202.66:34473)
0810 22:39:22.880894 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.880907 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.881230 (+ 323us) server_negotiation.cc:407] Connection header received
0810 22:39:22.881496 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.881503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.881706 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.881838 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.883899 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.883908 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.883918 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.883933 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.883988 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.884031 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.884115 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.884328 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.884408 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":106,"threads_started":1}
I20260810 22:39:22.884655 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.881520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.19.202.65:43215)
0810 22:39:22.882078 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:22.882091 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:22.882119 (+ 28us) server_negotiation.cc:407] Connection header received
0810 22:39:22.882179 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:22.882186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:22.882343 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:22.882491 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:22.883593 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:22.883599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:22.883637 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:22.883651 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:22.883695 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:22.883733 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:22.883835 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:22.884404 (+ 569us) server_negotiation.cc:299] Negotiation successful
0810 22:39:22.884500 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":229,"threads_started":1}
I20260810 22:39:22.885026 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.880209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:34473 (local address 127.0.0.1:45766)
0810 22:39:22.881074 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.881128 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.881339 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.882549 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.882552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.882568 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:22.883684 (+ 1116us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:22.883687 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:22.883699 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:22.883729 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:22.884194 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:22.884198 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:22.884201 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.884840 (+ 639us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.884878 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":96,"threads_started":1}
I20260810 22:39:22.885126 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:22.880209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:43935 (local address 127.0.0.1:45250)
0810 22:39:22.881302 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:22.881317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:22.881460 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:22.882477 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:22.882483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:22.882560 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:22.883326 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:22.883330 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:22.883353 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:22.883423 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:22.883921 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:22.883925 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:22.883931 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 22:39:22.884615 (+ 684us) client_negotiation.cc:240] Negotiation successful
0810 22:39:22.884658 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"mutex_wait_us":498,"thread_start_us":116,"threads_started":1}
I20260810 22:39:22.887769 25530 tablet_service.cc:1467] Processing CreateTablet for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b (DEFAULT_TABLE table=test-table [id=9eded98a8d054b8b8c52c1d983f3e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:22.887768 25677 tablet_service.cc:1467] Processing CreateTablet for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b (DEFAULT_TABLE table=test-table [id=9eded98a8d054b8b8c52c1d983f3e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:22.888921 25677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:22.888921 25530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:22.890434 25816 tablet_service.cc:1467] Processing CreateTablet for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b (DEFAULT_TABLE table=test-table [id=9eded98a8d054b8b8c52c1d983f3e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:22.891499 25816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:22.896673 25904 tablet_bootstrap.cc:492] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc: Bootstrap starting.
I20260810 22:39:22.897136 25905 tablet_bootstrap.cc:492] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Bootstrap starting.
I20260810 22:39:22.897830 25908 tablet_bootstrap.cc:492] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Bootstrap starting.
I20260810 22:39:22.899601 25905 tablet_bootstrap.cc:654] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:22.899758 25908 tablet_bootstrap.cc:654] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:22.900238 25904 tablet_bootstrap.cc:654] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:22.900516 25908 log.cc:824] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:22.900532 25905 log.cc:824] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:22.901095 25904 log.cc:824] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:22.902544 25905 tablet_bootstrap.cc:492] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: No bootstrap required, opened a new log
I20260810 22:39:22.902635 25908 tablet_bootstrap.cc:492] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: No bootstrap required, opened a new log
I20260810 22:39:22.902774 25905 ts_tablet_manager.cc:1397] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 22:39:22.902822 25908 ts_tablet_manager.cc:1397] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260810 22:39:22.903072 25904 tablet_bootstrap.cc:492] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc: No bootstrap required, opened a new log
I20260810 22:39:22.903244 25904 ts_tablet_manager.cc:1397] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 22:39:22.906239 25905 raft_consensus.cc:348] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:22.906490 25905 raft_consensus.cc:374] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:22.906570 25905 raft_consensus.cc:729] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc451e494b346469adcc6fa2870ed88, State: Initialized, Role: FOLLOWER
I20260810 22:39:22.907038 25905 consensus_queue.cc:260] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:22.907362 25908 raft_consensus.cc:348] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:22.907462 25904 raft_consensus.cc:348] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:22.907687 25904 raft_consensus.cc:374] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:22.907677 25908 raft_consensus.cc:374] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:22.907707 25746 heartbeater.cc:503] Master 127.19.202.126:36611 was elected leader, sending a full tablet report...
I20260810 22:39:22.907759 25904 raft_consensus.cc:729] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5653bb2b3b74a89a99bd3695bc02ecc, State: Initialized, Role: FOLLOWER
I20260810 22:39:22.907757 25908 raft_consensus.cc:729] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d76113cbd9e48e59f0a74e7efd4a7e5, State: Initialized, Role: FOLLOWER
I20260810 22:39:22.908104 25905 ts_tablet_manager.cc:1428] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:39:22.908143 25904 consensus_queue.cc:260] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:22.908715 25908 consensus_queue.cc:260] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:22.909137 25904 ts_tablet_manager.cc:1428] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:39:22.909202 25606 heartbeater.cc:503] Master 127.19.202.126:36611 was elected leader, sending a full tablet report...
I20260810 22:39:22.909735 25908 ts_tablet_manager.cc:1428] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 22:39:22.910043 25886 heartbeater.cc:503] Master 127.19.202.126:36611 was elected leader, sending a full tablet report...
W20260810 22:39:22.924770 25747 tablet.cc:2369] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:39:23.008347 25607 tablet.cc:2369] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:39:23.103153 25887 tablet.cc:2369] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:39:23.193073 25913 raft_consensus.cc:482] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:23.193383 25913 raft_consensus.cc:504] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:23.194788 25913 leader_election.cc:288] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bc451e494b346469adcc6fa2870ed88 (127.19.202.66:34473), 2d76113cbd9e48e59f0a74e7efd4a7e5 (127.19.202.67:43935)
I20260810 22:39:23.197067 25899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.195213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:47553 (local address 127.19.202.67:43935)
0810 22:39:23.195404 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:23.195411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:23.195532 (+ 121us) server_negotiation.cc:407] Connection header received
0810 22:39:23.195717 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:23.195722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:23.195770 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:23.195858 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:23.196647 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:23.196650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:23.196651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:23.196662 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:23.196681 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:23.196688 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:23.196809 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:23.196875 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 22:39:23.196919 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:39:23.197083 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.195050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:43935 (local address 127.19.202.65:47553)
0810 22:39:23.195394 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:23.195409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:23.195569 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:23.195900 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:23.195903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:23.195926 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:23.196458 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:23.196459 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:23.196469 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:23.196493 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:23.196768 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:23.196771 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:23.196794 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 22:39:23.196882 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 22:39:23.196940 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":91,"threads_started":1}
I20260810 22:39:23.197326 25900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.195213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:52823 (local address 127.19.202.66:34473)
0810 22:39:23.195388 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:23.195392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:23.195643 (+ 251us) server_negotiation.cc:407] Connection header received
0810 22:39:23.195785 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:23.195787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:23.195818 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:23.195907 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:23.196887 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:23.196891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:23.196894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:23.196907 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:23.196938 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:23.196956 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:23.197058 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:23.197144 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 22:39:23.197186 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 22:39:23.197355 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.195050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:34473 (local address 127.19.202.65:52823)
0810 22:39:23.195536 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:23.195556 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:23.195680 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:23.195959 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:23.195963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:23.195984 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:23.196663 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:23.196665 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:23.196680 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:23.196714 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:23.197055 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:23.197058 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:23.197063 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:39:23.197150 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 22:39:23.197186 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":20,"thread_start_us":57,"threads_started":1}
I20260810 22:39:23.197731 25837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fbcc4e9cea485c9a67f3d432bdbe4b" candidate_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" is_pre_election: true
I20260810 22:39:23.198030 25837 raft_consensus.cc:2454] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5653bb2b3b74a89a99bd3695bc02ecc in term 0.
I20260810 22:39:23.198087 25697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fbcc4e9cea485c9a67f3d432bdbe4b" candidate_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc451e494b346469adcc6fa2870ed88" is_pre_election: true
I20260810 22:39:23.198406 25697 raft_consensus.cc:2454] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5653bb2b3b74a89a99bd3695bc02ecc in term 0.
I20260810 22:39:23.198531 25483 leader_election.cc:302] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d76113cbd9e48e59f0a74e7efd4a7e5, b5653bb2b3b74a89a99bd3695bc02ecc; no voters:
I20260810 22:39:23.198824 25913 raft_consensus.cc:2781] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:23.198938 25913 raft_consensus.cc:482] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:23.199007 25913 raft_consensus.cc:3037] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:23.200166 25913 raft_consensus.cc:504] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:23.200572 25913 leader_election.cc:288] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [CANDIDATE]: Term 1 election: Requested vote from peers 1bc451e494b346469adcc6fa2870ed88 (127.19.202.66:34473), 2d76113cbd9e48e59f0a74e7efd4a7e5 (127.19.202.67:43935)
I20260810 22:39:23.200917 25697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fbcc4e9cea485c9a67f3d432bdbe4b" candidate_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc451e494b346469adcc6fa2870ed88"
I20260810 22:39:23.200925 25837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fbcc4e9cea485c9a67f3d432bdbe4b" candidate_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5"
I20260810 22:39:23.201027 25697 raft_consensus.cc:3037] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:23.201052 25837 raft_consensus.cc:3037] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:23.202090 25837 raft_consensus.cc:2454] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5653bb2b3b74a89a99bd3695bc02ecc in term 1.
I20260810 22:39:23.202165 25697 raft_consensus.cc:2454] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5653bb2b3b74a89a99bd3695bc02ecc in term 1.
I20260810 22:39:23.202399 25483 leader_election.cc:302] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d76113cbd9e48e59f0a74e7efd4a7e5, b5653bb2b3b74a89a99bd3695bc02ecc; no voters:
I20260810 22:39:23.202611 25913 raft_consensus.cc:2781] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:23.202860 25913 raft_consensus.cc:686] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [term 1 LEADER]: Becoming Leader. State: Replica: b5653bb2b3b74a89a99bd3695bc02ecc, State: Running, Role: LEADER
I20260810 22:39:23.203197 25913 consensus_queue.cc:237] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } }
I20260810 22:39:23.206305 25395 catalog_manager.cc:5495] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc reported cstate change: term changed from 0 to 1, leader changed from <none> to b5653bb2b3b74a89a99bd3695bc02ecc (127.19.202.65). New cstate: current_term: 1 leader_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5653bb2b3b74a89a99bd3695bc02ecc" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 43215 } health_report { overall_health: HEALTHY } } }
I20260810 22:39:23.245503 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.243812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34812 (local address 127.19.202.65:43215)
0810 22:39:23.243957 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:23.243962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:23.243978 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:39:23.244092 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:23.244095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:23.244159 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:23.244258 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:23.245076 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:23.245080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:23.245082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:23.245093 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:23.245115 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:23.245130 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:23.245220 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:23.245284 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 22:39:23.245326 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:39:23.251618 25837 raft_consensus.cc:1261] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 1 FOLLOWER]: Refusing update from remote peer b5653bb2b3b74a89a99bd3695bc02ecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:23.251618 25697 raft_consensus.cc:1261] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 1 FOLLOWER]: Refusing update from remote peer b5653bb2b3b74a89a99bd3695bc02ecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:23.252413 25918 consensus_queue.cc:1035] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d76113cbd9e48e59f0a74e7efd4a7e5" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 43935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:23.252614 25913 consensus_queue.cc:1035] T 34fbcc4e9cea485c9a67f3d432bdbe4b P b5653bb2b3b74a89a99bd3695bc02ecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc451e494b346469adcc6fa2870ed88" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 34473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:23.271260 25396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:23.271420 25396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:23.273792 25396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ee4b11db760489ab9bcad3b4bd44beb: Sending DeleteTablet for 3 replicas of tablet 34fbcc4e9cea485c9a67f3d432bdbe4b
I20260810 22:39:23.274668 25677 tablet_service.cc:1514] Processing DeleteTablet for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:23 UTC) from {username='slave'} at 127.0.0.1:45766
I20260810 22:39:23.274668 25530 tablet_service.cc:1514] Processing DeleteTablet for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:23 UTC) from {username='slave'} at 127.0.0.1:34796
I20260810 22:39:23.274873 25816 tablet_service.cc:1514] Processing DeleteTablet for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:23 UTC) from {username='slave'} at 127.0.0.1:45250
I20260810 22:39:23.275364 25934 tablet_replica.cc:330] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: stopping tablet replica
I20260810 22:39:23.275478 25932 tablet_replica.cc:330] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: stopping tablet replica
I20260810 22:39:23.275755 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 25464
I20260810 22:39:23.275789 25932 raft_consensus.cc:2229] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:23.275789 25934 raft_consensus.cc:2229] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:23.276113 25934 raft_consensus.cc:2258] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:23.276114 25932 raft_consensus.cc:2258] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:23.279119 25932 ts_tablet_manager.cc:1905] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:23.279786 25934 ts_tablet_manager.cc:1905] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:23.281566 25932 ts_tablet_manager.cc:1918] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:23.281679 25932 log.cc:1196] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/34fbcc4e9cea485c9a67f3d432bdbe4b
I20260810 22:39:23.281982 25932 ts_tablet_manager.cc:1939] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 1bc451e494b346469adcc6fa2870ed88: Deleting consensus metadata
I20260810 22:39:23.283206 25934 ts_tablet_manager.cc:1918] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:23.283316 25934 log.cc:1196] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/34fbcc4e9cea485c9a67f3d432bdbe4b
I20260810 22:39:23.283329 25382 catalog_manager.cc:4841] TS 1bc451e494b346469adcc6fa2870ed88 (127.19.202.66:34473): tablet 34fbcc4e9cea485c9a67f3d432bdbe4b (table test-table [id=9eded98a8d054b8b8c52c1d983f3e837]) successfully deleted
I20260810 22:39:23.283706 25934 ts_tablet_manager.cc:1939] T 34fbcc4e9cea485c9a67f3d432bdbe4b P 2d76113cbd9e48e59f0a74e7efd4a7e5: Deleting consensus metadata
I20260810 22:39:23.284950 25383 catalog_manager.cc:4841] TS 2d76113cbd9e48e59f0a74e7efd4a7e5 (127.19.202.67:43935): tablet 34fbcc4e9cea485c9a67f3d432bdbe4b (table test-table [id=9eded98a8d054b8b8c52c1d983f3e837]) successfully deleted
W20260810 22:39:23.286644 25382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:43215 (error 108)
I20260810 22:39:23.286747 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 25609
I20260810 22:39:23.289601 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.289196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:43215 (local address 127.0.0.1:34820)
0810 22:39:23.289431 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:23.289513 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:43215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260810 22:39:23.289855 25382 catalog_manager.cc:4584] TS b5653bb2b3b74a89a99bd3695bc02ecc (127.19.202.65:43215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 34fbcc4e9cea485c9a67f3d432bdbe4b: Network error: Client connection negotiation failed: client connection to 127.19.202.65:43215: connect: Connection refused (error 111)
I20260810 22:39:23.294901 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 25752
I20260810 22:39:23.302831 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 25364
2026-08-10T22:39:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1204 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 22:39:23.327922 20265 mini_oidc.cc:152] Starting JWKS server
I20260810 22:39:23.329749 20265 webserver.cc:460] Webserver started at https://127.0.0.1:37487/ using document root <none> and password file <none>
I20260810 22:39:23.329941 20265 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 22:39:23.330245 20265 webserver.cc:460] Webserver started at http://127.0.0.1:32829/ using document root <none> and password file <none>
2026-08-10T22:39:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:23Z Disabled control of system clock
I20260810 22:39:23.340768 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:40357
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:38837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:40357
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37487/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/testchainca.cert with env {}
W20260810 22:39:23.452080 25941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:23.452346 25941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:23.452383 25941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:23.456328 25941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:23.456386 25941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:23.456403 25941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:23.456421 25941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:23.456436 25941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:39:23.456451 25941 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:39:23.456465 25941 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37487/jwks.json
I20260810 22:39:23.461735 25941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:38837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:40357
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:40357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37487/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:23.462780 25941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:23.463780 25941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:39:23.472460 25941 server_base.cc:1034] running on GCE node
W20260810 22:39:23.472498 25949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:23.472404 25948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:23.472419 25951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:23.473449 25941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:23.474542 25941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:23.475736 25941 hybrid_clock.cc:648] HybridClock initialized: now 1786401563475705 us; error 43 us; skew 500 ppm
I20260810 22:39:23.477876 25941 webserver.cc:460] Webserver started at http://127.19.202.126:33239/ using document root <none> and password file <none>
I20260810 22:39:23.478520 25941 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:23.478592 25941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:23.478868 25941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:23.481112 25941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "12e44f8f86bf4ae4ad24f1092b426a60"
format_stamp: "Formatted at 2026-08-10 22:39:23 on dist-test-slave-1x4g"
server_key: "02c6cd3cc71253a51088c920efd21931"
server_key_iv: "850e7f2dbbfe9d2d176848f0562beeff"
server_key_version: "encryptionkey@0"
I20260810 22:39:23.481763 25941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "12e44f8f86bf4ae4ad24f1092b426a60"
format_stamp: "Formatted at 2026-08-10 22:39:23 on dist-test-slave-1x4g"
server_key: "02c6cd3cc71253a51088c920efd21931"
server_key_iv: "850e7f2dbbfe9d2d176848f0562beeff"
server_key_version: "encryptionkey@0"
I20260810 22:39:23.485841 25941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 22:39:23.488300 25957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:23.489522 25941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:23.489658 25941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "12e44f8f86bf4ae4ad24f1092b426a60"
format_stamp: "Formatted at 2026-08-10 22:39:23 on dist-test-slave-1x4g"
server_key: "02c6cd3cc71253a51088c920efd21931"
server_key_iv: "850e7f2dbbfe9d2d176848f0562beeff"
server_key_version: "encryptionkey@0"
I20260810 22:39:23.489766 25941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:23.507761 25941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:23.508507 25941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:23.508755 25941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:23.516716 25941 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:40357
I20260810 22:39:23.516726 26012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:40357 every 8 connection(s)
I20260810 22:39:23.517803 25941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:23.520895 26013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:23.526738 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 25941
I20260810 22:39:23.526861 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:23.526906 26013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60: Bootstrap starting.
I20260810 22:39:23.527140 20265 external_mini_cluster.cc:1428] Setting key 28ece716ed38798f3aa2e30ac5f8331b
I20260810 22:39:23.529508 26013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:23.530344 26013 log.cc:824] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:23.532316 26013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60: No bootstrap required, opened a new log
I20260810 22:39:23.536500 26013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40357 } }
I20260810 22:39:23.536687 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.528423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.19.202.126:40357)
0810 22:39:23.529076 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:23.529090 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:23.529124 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:23.529857 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:23.529879 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:23.530437 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:23.530796 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:23.531277 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.532205 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.533453 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.533923 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.534043 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:23.534873 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:23.534879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:23.534895 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:23.535266 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:23.535317 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:23.535336 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:23.535416 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:23.535638 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 22:39:23.535932 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":140,"threads_started":1}
I20260810 22:39:23.536722 26013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:23.536856 26013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e44f8f86bf4ae4ad24f1092b426a60, State: Initialized, Role: FOLLOWER
I20260810 22:39:23.537338 26013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40357 } }
I20260810 22:39:23.537495 26013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:23.537585 26013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:23.537705 26013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:23.538574 26013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40357 } }
I20260810 22:39:23.538940 26013 leader_election.cc:302] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e44f8f86bf4ae4ad24f1092b426a60; no voters:
I20260810 22:39:23.539270 26013 leader_election.cc:288] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:23.539346 26019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:23.539620 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:39:23.539597 26019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [term 1 LEADER]: Becoming Leader. State: Replica: 12e44f8f86bf4ae4ad24f1092b426a60, State: Running, Role: LEADER
I20260810 22:39:23.539937 26019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40357 } }
I20260810 22:39:23.540596 26013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:23.541118 26020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40357 } } }
I20260810 22:39:23.541121 26019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e44f8f86bf4ae4ad24f1092b426a60. Latest consensus state: current_term: 1 leader_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e44f8f86bf4ae4ad24f1092b426a60" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40357 } } }
I20260810 22:39:23.541299 26020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:23.541301 26019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:23.541781 26024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:23.545327 26024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:23.550673 26024 catalog_manager.cc:1347] Generated new cluster ID: 42adce8f30f54a6ea3472f86498bf471
I20260810 22:39:23.550729 26024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:23.561343 26024 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:23.562103 26024 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:23.570405 26024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12e44f8f86bf4ae4ad24f1092b426a60: Generated new TSK 0
I20260810 22:39:23.571027 26024 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 22:39:23.606729 26047 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 22:39:23.606861 26047 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 22:39:23.607600 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.582861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.19.202.126:40357)
0810 22:39:23.583002 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:23.583008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:23.583315 (+ 307us) server_negotiation.cc:407] Connection header received
0810 22:39:23.583539 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:23.583541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:23.583585 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:23.583666 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:39:23.583952 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.584502 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.585729 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.585950 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.586031 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:23.586396 (+ 365us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:39:23.607151 (+ 20755us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 22:39:23.607447 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49276: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 22:39:23.607685 26017 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49276: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 22:39:23.608950 26046 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:23.582801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:40357 (local address 127.0.0.1:49276)
0810 22:39:23.583223 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:23.583238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:23.583389 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:23.583705 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:23.583709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:23.583718 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:39:23.583835 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:23.583842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.584619 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.584623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:23.585600 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:23.585610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.586069 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.586073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:23.586167 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:23.586220 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:39:23.607385 (+ 21165us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 22:39:23.607587 (+ 202us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.202.126:40357: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":148,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.202.126:40357: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 22:39:23.609779 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 25941
2026-08-10T22:39:23Z chronyd exiting
I20260810 22:39:23.650882 20265 test_util.cc:175] -----------------------------------------------
I20260810 22:39:23.650982 20265 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786401536977653-20265-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (324 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 22:39:23.652091 20265 mini_oidc.cc:152] Starting JWKS server
I20260810 22:39:23.653029 20265 webserver.cc:460] Webserver started at https://127.0.0.1:42091/ using document root <none> and password file <none>
I20260810 22:39:23.653117 20265 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 22:39:23.653333 20265 webserver.cc:460] Webserver started at http://127.0.0.1:40159/ using document root <none> and password file <none>
2026-08-10T22:39:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:23Z Disabled control of system clock
I20260810 22:39:23.664139 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36891
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:36891
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42091/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/testchainca.cert with env {}
W20260810 22:39:23.776063 26053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:23.776300 26053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:23.776337 26053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:23.780318 26053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:23.780366 26053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:23.780380 26053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:23.780395 26053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:23.780428 26053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:39:23.780462 26053 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:39:23.780478 26053 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42091/jwks.json
I20260810 22:39:23.785874 26053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:36891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42091/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:23.786972 26053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:23.788017 26053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:23.796654 26061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:23.796654 26058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:23.796662 26059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:23.797144 26053 server_base.cc:1034] running on GCE node
I20260810 22:39:23.797701 26053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:23.798766 26053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:23.799971 26053 hybrid_clock.cc:648] HybridClock initialized: now 1786401563799954 us; error 30 us; skew 500 ppm
I20260810 22:39:23.801858 26053 webserver.cc:460] Webserver started at http://127.19.202.126:36465/ using document root <none> and password file <none>
I20260810 22:39:23.802448 26053 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:23.802542 26053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:23.802799 26053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:23.804875 26053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "01cd5627a486427ca9b0268b90704f9e"
format_stamp: "Formatted at 2026-08-10 22:39:23 on dist-test-slave-1x4g"
server_key: "f5062392147950298389b4433974fe41"
server_key_iv: "97f3b4d197a734822d762876dc1f0544"
server_key_version: "encryptionkey@0"
I20260810 22:39:23.805465 26053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "01cd5627a486427ca9b0268b90704f9e"
format_stamp: "Formatted at 2026-08-10 22:39:23 on dist-test-slave-1x4g"
server_key: "f5062392147950298389b4433974fe41"
server_key_iv: "97f3b4d197a734822d762876dc1f0544"
server_key_version: "encryptionkey@0"
I20260810 22:39:23.808938 26053 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 22:39:23.811241 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:23.812463 26053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:23.812618 26053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "01cd5627a486427ca9b0268b90704f9e"
format_stamp: "Formatted at 2026-08-10 22:39:23 on dist-test-slave-1x4g"
server_key: "f5062392147950298389b4433974fe41"
server_key_iv: "97f3b4d197a734822d762876dc1f0544"
server_key_version: "encryptionkey@0"
I20260810 22:39:23.812743 26053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:23.825322 26053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:23.825915 26053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:23.826102 26053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:23.832945 26053 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:36891
I20260810 22:39:23.832954 26125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:36891 every 8 connection(s)
I20260810 22:39:23.833871 26053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:23.837065 26126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:23.840096 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26053
I20260810 22:39:23.840191 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:23.840462 20265 external_mini_cluster.cc:1428] Setting key df2c09b83e537a03a9a39e69135ed46b
I20260810 22:39:23.842751 26126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e: Bootstrap starting.
I20260810 22:39:23.845186 26126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:23.845925 26126 log.cc:824] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:23.847718 26126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e: No bootstrap required, opened a new log
I20260810 22:39:23.849402 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.841805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.19.202.126:36891)
0810 22:39:23.842427 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:23.842443 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:23.842478 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:39:23.843139 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:23.843157 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:23.843557 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:23.843813 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:23.844177 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.845195 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.846070 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.846717 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.846879 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:23.847719 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:23.847725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:23.847746 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:23.848145 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:23.848196 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:23.848214 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:23.848315 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:23.848428 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 22:39:23.848690 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":148,"threads_started":1}
I20260810 22:39:23.851569 26126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cd5627a486427ca9b0268b90704f9e" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36891 } }
I20260810 22:39:23.851863 26126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:23.851927 26126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01cd5627a486427ca9b0268b90704f9e, State: Initialized, Role: FOLLOWER
I20260810 22:39:23.852185 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:39:23.852391 26126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cd5627a486427ca9b0268b90704f9e" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36891 } }
I20260810 22:39:23.852545 26126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:23.852608 26126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:23.852702 26126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:23.853859 26126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cd5627a486427ca9b0268b90704f9e" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36891 } }
I20260810 22:39:23.854229 26126 leader_election.cc:302] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01cd5627a486427ca9b0268b90704f9e; no voters:
I20260810 22:39:23.854532 26126 leader_election.cc:288] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:23.854614 26132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:23.854879 26132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [term 1 LEADER]: Becoming Leader. State: Replica: 01cd5627a486427ca9b0268b90704f9e, State: Running, Role: LEADER
I20260810 22:39:23.855193 26132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cd5627a486427ca9b0268b90704f9e" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36891 } }
I20260810 22:39:23.855789 26126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:23.856997 26134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cd5627a486427ca9b0268b90704f9e. Latest consensus state: current_term: 1 leader_uuid: "01cd5627a486427ca9b0268b90704f9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cd5627a486427ca9b0268b90704f9e" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36891 } } }
I20260810 22:39:23.857151 26134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:23.857458 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01cd5627a486427ca9b0268b90704f9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01cd5627a486427ca9b0268b90704f9e" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36891 } } }
I20260810 22:39:23.857563 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:23.857630 26142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:23.860567 26142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:23.866063 26142 catalog_manager.cc:1347] Generated new cluster ID: 6e6fc417b0174722bbd39eb3e496adba
I20260810 22:39:23.866127 26142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:23.876488 26142 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:23.877380 26142 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:23.881124 26142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01cd5627a486427ca9b0268b90704f9e: Generated new TSK 0
I20260810 22:39:23.881851 26142 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 22:39:23.922359 26159 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 22:39:23.922489 26159 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 22:39:23.923247 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:23.899822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43820 (local address 127.19.202.126:36891)
0810 22:39:23.899963 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:23.899967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:23.900161 (+ 194us) server_negotiation.cc:407] Connection header received
0810 22:39:23.900362 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:23.900364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:23.900410 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:23.900547 (+ 137us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:39:23.900845 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.901361 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.902256 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.902495 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.902608 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:23.902925 (+ 317us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:39:23.922759 (+ 19834us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42091/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 22:39:23.923088 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43820: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42091/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 22:39:23.923336 26129 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43820: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42091/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 22:39:23.923395 26158 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:23.899760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36891 (local address 127.0.0.1:43820)
0810 22:39:23.900068 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:23.900083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:23.900211 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:23.900549 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:23.900552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:23.900560 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:39:23.900702 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:23.900709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.901489 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.901493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:23.902118 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:23.902126 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:23.902635 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:23.902638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:23.902729 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:23.902751 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:39:23.923005 (+ 20254us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42091/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 22:39:23.923200 (+ 195us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.202.126:36891: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42091/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":112,"threads_started":1}
I20260810 22:39:23.924044 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26053
2026-08-10T22:39:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (314 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 22:39:23.966336 20265 mini_oidc.cc:152] Starting JWKS server
I20260810 22:39:23.967223 20265 webserver.cc:460] Webserver started at https://127.0.0.1:44669/ using document root <none> and password file <none>
I20260810 22:39:23.967341 20265 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 22:39:23.967658 20265 webserver.cc:460] Webserver started at http://127.0.0.1:44171/ using document root <none> and password file <none>
2026-08-10T22:39:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:23Z Disabled control of system clock
I20260810 22:39:23.978509 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41571
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:41571
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44669/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 22:39:24.098062 26165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:24.098400 26165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:24.098511 26165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:24.102640 26165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:24.102722 26165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:24.102767 26165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:24.102838 26165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:24.102898 26165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:39:24.102941 26165 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:39:24.102977 26165 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44669/jwks.json
I20260810 22:39:24.107797 26165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:41571
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44669/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:24.108994 26165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:24.110394 26165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:24.119210 26171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:24.119218 26170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:24.119218 26174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:24.119267 26165 server_base.cc:1034] running on GCE node
I20260810 22:39:24.120131 26165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:24.121245 26165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:24.122465 26165 hybrid_clock.cc:648] HybridClock initialized: now 1786401564122449 us; error 36 us; skew 500 ppm
I20260810 22:39:24.124231 26165 webserver.cc:460] Webserver started at http://127.19.202.126:37641/ using document root <none> and password file <none>
I20260810 22:39:24.124850 26165 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:24.124941 26165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:24.125177 26165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:24.127251 26165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "31683419c55b40f4a5111f0cc0037da4"
format_stamp: "Formatted at 2026-08-10 22:39:24 on dist-test-slave-1x4g"
server_key: "14e805036941149553b9562599295f06"
server_key_iv: "f5a6c45acd04adc65ee95298e9f93d33"
server_key_version: "encryptionkey@0"
I20260810 22:39:24.127831 26165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "31683419c55b40f4a5111f0cc0037da4"
format_stamp: "Formatted at 2026-08-10 22:39:24 on dist-test-slave-1x4g"
server_key: "14e805036941149553b9562599295f06"
server_key_iv: "f5a6c45acd04adc65ee95298e9f93d33"
server_key_version: "encryptionkey@0"
I20260810 22:39:24.131562 26165 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 22:39:24.134044 26180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:24.135247 26165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:24.135421 26165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "31683419c55b40f4a5111f0cc0037da4"
format_stamp: "Formatted at 2026-08-10 22:39:24 on dist-test-slave-1x4g"
server_key: "14e805036941149553b9562599295f06"
server_key_iv: "f5a6c45acd04adc65ee95298e9f93d33"
server_key_version: "encryptionkey@0"
I20260810 22:39:24.135540 26165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:24.147969 26165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:24.148607 26165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:24.148818 26165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:24.155989 26165 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:41571
I20260810 22:39:24.156005 26235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:41571 every 8 connection(s)
I20260810 22:39:24.156930 26165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:24.160080 26236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:24.164345 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26165
I20260810 22:39:24.164470 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:24.164767 20265 external_mini_cluster.cc:1428] Setting key 3ec22f29436b3ebf79937c0fb303752c
I20260810 22:39:24.166054 26236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4: Bootstrap starting.
I20260810 22:39:24.168318 26236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:24.169124 26236 log.cc:824] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:24.170827 26236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4: No bootstrap required, opened a new log
I20260810 22:39:24.174065 26236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31683419c55b40f4a5111f0cc0037da4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41571 } }
I20260810 22:39:24.174309 26236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:24.174384 26236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31683419c55b40f4a5111f0cc0037da4, State: Initialized, Role: FOLLOWER
I20260810 22:39:24.174454 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:24.166121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.19.202.126:41571)
0810 22:39:24.166745 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:24.166759 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:24.166793 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:24.167449 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:24.167465 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:24.167938 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:24.168274 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:24.168700 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.169742 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.170614 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.171145 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.171320 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:24.172161 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:24.172166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:24.172178 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:24.172567 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:24.172617 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:24.172636 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:24.172746 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:24.172902 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 22:39:24.173139 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":126,"threads_started":1}
I20260810 22:39:24.174877 26236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31683419c55b40f4a5111f0cc0037da4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41571 } }
I20260810 22:39:24.175045 26236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:24.175129 26236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:24.175226 26236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:24.177254 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:39:24.201352 26236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31683419c55b40f4a5111f0cc0037da4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41571 } }
I20260810 22:39:24.201745 26236 leader_election.cc:302] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31683419c55b40f4a5111f0cc0037da4; no voters:
I20260810 22:39:24.202087 26236 leader_election.cc:288] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:24.202207 26243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:24.202478 26243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [term 1 LEADER]: Becoming Leader. State: Replica: 31683419c55b40f4a5111f0cc0037da4, State: Running, Role: LEADER
I20260810 22:39:24.202944 26243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31683419c55b40f4a5111f0cc0037da4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41571 } }
I20260810 22:39:24.203454 26236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:24.204684 26244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31683419c55b40f4a5111f0cc0037da4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31683419c55b40f4a5111f0cc0037da4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41571 } } }
I20260810 22:39:24.204856 26244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:24.205238 26245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31683419c55b40f4a5111f0cc0037da4. Latest consensus state: current_term: 1 leader_uuid: "31683419c55b40f4a5111f0cc0037da4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31683419c55b40f4a5111f0cc0037da4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41571 } } }
I20260810 22:39:24.205384 26245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4 [sys.catalog]: This master's current role is: LEADER
W20260810 22:39:24.208429 26259 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:39:24.208491 26259 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:39:24.208565 26253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:24.209463 26253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:24.215185 26253 catalog_manager.cc:1347] Generated new cluster ID: d30f8d918b974c81896d470980ee1468
I20260810 22:39:24.215247 26253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:24.237085 26253 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:24.238308 26253 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:24.244623 26253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31683419c55b40f4a5111f0cc0037da4: Generated new TSK 0
I20260810 22:39:24.245401 26253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:39:24.268970 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:24.261825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41338 (local address 127.19.202.126:41571)
0810 22:39:24.261964 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:24.261969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:24.262158 (+ 189us) server_negotiation.cc:407] Connection header received
0810 22:39:24.262309 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:24.262311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:24.262374 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:24.262461 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:39:24.262728 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.263306 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.264530 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.264742 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.264859 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:24.265152 (+ 293us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:39:24.268579 (+ 3427us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 22:39:24.268827 (+ 248us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41338: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 22:39:24.269040 26241 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41338: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 22:39:24.270956 26268 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:24.261766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:41571 (local address 127.0.0.1:41338)
0810 22:39:24.262067 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:24.262082 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:24.262194 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:24.262482 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:24.262486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:24.262493 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:39:24.262612 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:24.262619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.263418 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.263422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:24.264412 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:24.264421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.264876 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.264880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:24.264974 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:24.265002 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:39:24.268745 (+ 3743us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 22:39:24.270747 (+ 2002us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.202.126:41571: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44669/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":1782,"thread_start_us":114,"threads_started":1}
I20260810 22:39:24.271440 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26165
E20260810 22:39:24.282253 26269 webserver.cc:533] Webserver: sslize failed:
E20260810 22:39:24.282310 26269 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T22:39:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (349 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T22:39:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:24Z Disabled control of system clock
I20260810 22:39:24.325765 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34259
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34259
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/testchainca.cert with env {}
W20260810 22:39:24.445858 26273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:24.446105 26273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:24.446172 26273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:24.450341 26273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:24.450402 26273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:24.450419 26273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:24.450464 26273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:24.450502 26273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:39:24.450542 26273 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:39:24.450598 26273 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 22:39:24.456125 26273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34259
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:24.457120 26273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:24.458099 26273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:24.466770 26278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:24.466840 26273 server_base.cc:1034] running on GCE node
W20260810 22:39:24.466799 26280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:24.466778 26282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:24.467489 26273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:24.468454 26273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:24.469645 26273 hybrid_clock.cc:648] HybridClock initialized: now 1786401564469598 us; error 48 us; skew 500 ppm
I20260810 22:39:24.471347 26273 webserver.cc:460] Webserver started at http://127.19.202.126:34287/ using document root <none> and password file <none>
I20260810 22:39:24.471879 26273 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:24.471938 26273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:24.472172 26273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:24.474182 26273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "f26fe92a51f244e5b200b2e2dbba95d6"
format_stamp: "Formatted at 2026-08-10 22:39:24 on dist-test-slave-1x4g"
server_key: "07cdb03eb66e73f0495a42f8466a9347"
server_key_iv: "e326a5138acfd09c993a5b5567803270"
server_key_version: "encryptionkey@0"
I20260810 22:39:24.474771 26273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "f26fe92a51f244e5b200b2e2dbba95d6"
format_stamp: "Formatted at 2026-08-10 22:39:24 on dist-test-slave-1x4g"
server_key: "07cdb03eb66e73f0495a42f8466a9347"
server_key_iv: "e326a5138acfd09c993a5b5567803270"
server_key_version: "encryptionkey@0"
I20260810 22:39:24.478425 26273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:39:24.480948 26289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:24.482147 26273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:24.482259 26273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "f26fe92a51f244e5b200b2e2dbba95d6"
format_stamp: "Formatted at 2026-08-10 22:39:24 on dist-test-slave-1x4g"
server_key: "07cdb03eb66e73f0495a42f8466a9347"
server_key_iv: "e326a5138acfd09c993a5b5567803270"
server_key_version: "encryptionkey@0"
I20260810 22:39:24.482362 26273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:24.505039 26273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:24.505693 26273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:24.505879 26273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:24.513289 26273 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:34259
I20260810 22:39:24.513304 26348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:34259 every 8 connection(s)
I20260810 22:39:24.514245 26273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:24.517167 26349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:24.521891 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26273
I20260810 22:39:24.522025 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:24.522342 20265 external_mini_cluster.cc:1428] Setting key 2de79a149c4459da637068d26c40b96d
I20260810 22:39:24.522893 26349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6: Bootstrap starting.
I20260810 22:39:24.525310 26349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:24.526180 26349 log.cc:824] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:24.528383 26349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6: No bootstrap required, opened a new log
I20260810 22:39:24.531538 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:24.523637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.19.202.126:34259)
0810 22:39:24.524288 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:24.524302 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:24.524336 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:24.524971 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:24.524987 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:24.525403 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:24.525672 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:24.526048 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.527105 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.528330 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.528855 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.528966 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:24.529842 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:24.529853 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:24.529869 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:24.530259 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:24.530311 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:24.530330 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:24.530411 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:24.530588 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 22:39:24.530853 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":278,"thread_start_us":163,"threads_started":1}
I20260810 22:39:24.532517 26349 raft_consensus.cc:348] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34259 } }
I20260810 22:39:24.532729 26349 raft_consensus.cc:374] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:24.532820 26349 raft_consensus.cc:729] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26fe92a51f244e5b200b2e2dbba95d6, State: Initialized, Role: FOLLOWER
I20260810 22:39:24.533275 26349 consensus_queue.cc:260] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34259 } }
I20260810 22:39:24.533428 26349 raft_consensus.cc:388] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:24.533504 26349 raft_consensus.cc:482] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:24.533615 26349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:24.534849 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:39:24.610268 26349 raft_consensus.cc:504] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34259 } }
I20260810 22:39:24.610886 26349 leader_election.cc:302] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f26fe92a51f244e5b200b2e2dbba95d6; no voters:
I20260810 22:39:24.611449 26349 leader_election.cc:288] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:24.611600 26354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:24.611835 26354 raft_consensus.cc:686] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [term 1 LEADER]: Becoming Leader. State: Replica: f26fe92a51f244e5b200b2e2dbba95d6, State: Running, Role: LEADER
I20260810 22:39:24.612224 26354 consensus_queue.cc:237] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34259 } }
I20260810 22:39:24.612978 26349 sys_catalog.cc:564] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:24.614303 26356 sys_catalog.cc:455] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f26fe92a51f244e5b200b2e2dbba95d6. Latest consensus state: current_term: 1 leader_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34259 } } }
I20260810 22:39:24.614476 26356 sys_catalog.cc:458] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:24.614343 26355 sys_catalog.cc:455] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26fe92a51f244e5b200b2e2dbba95d6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34259 } } }
I20260810 22:39:24.614746 26355 sys_catalog.cc:458] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:24.614933 26363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:24.618400 26363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:24.623661 26363 catalog_manager.cc:1347] Generated new cluster ID: 0343dd4bcd144349828f6de5c68a9c1d
I20260810 22:39:24.623726 26363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:24.635851 26363 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:24.637090 26363 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:24.649252 26363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f26fe92a51f244e5b200b2e2dbba95d6: Generated new TSK 0
I20260810 22:39:24.650051 26363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:39:24.681095 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:24.658612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.19.202.126:34259)
0810 22:39:24.658752 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:24.658757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:24.658993 (+ 236us) server_negotiation.cc:407] Connection header received
0810 22:39:24.659137 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:24.659140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:24.659218 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:24.659336 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:39:24.659529 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.660376 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.661338 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.661701 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.661815 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:24.662130 (+ 315us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:39:24.680505 (+ 18375us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 22:39:24.680943 (+ 438us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59080: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 22:39:24.681180 26352 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59080: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 22:39:24.681151 26378 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:24.658535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34259 (local address 127.0.0.1:59080)
0810 22:39:24.658903 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:24.658915 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:24.659013 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:24.659341 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:24.659343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:24.659350 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:39:24.659445 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:24.659449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.660504 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.660507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:24.661214 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:24.661223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:24.661842 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:24.661845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:24.661935 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:24.661959 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:39:24.680751 (+ 18792us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 22:39:24.680999 (+ 248us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.202.126:34259: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":119,"threads_started":1}
I20260810 22:39:24.682006 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26273
2026-08-10T22:39:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (388 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (135 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:25 dist-test-slave-1x4g krb5kdc[26391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:25 dist-test-slave-1x4g krb5kdc[26391](info): set up 2 sockets
Aug 10 22:39:25 dist-test-slave-1x4g krb5kdc[26391](info): commencing operation
krb5kdc: starting...
W20260810 22:39:26.814873 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.846s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:26 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:26.938021 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41691
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:41691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:27.049288 26406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:27.049521 26406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:27.049558 26406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:27.053301 26406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:27.053352 26406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:27.053372 26406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:27.053409 26406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:27.053444 26406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:27.058494 26406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:41691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:27.059454 26406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:27.060469 26406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:27.068936 26415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:27.068974 26411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:27.068974 26412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:27.068989 26406 server_base.cc:1034] running on GCE node
I20260810 22:39:27.069933 26406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:27.071000 26406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:27.072209 26406 hybrid_clock.cc:648] HybridClock initialized: now 1786401567072179 us; error 42 us; skew 500 ppm
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.075032 26406 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:27.075929 26406 webserver.cc:460] Webserver started at http://127.19.202.126:45869/ using document root <none> and password file <none>
I20260810 22:39:27.076478 26406 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:27.076589 26406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:27.076924 26406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:27.078935 26406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "c25c079023a4c6f7643f1b1151f1152b"
server_key_iv: "d33d30c590052e70ac617913546d68e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.079535 26406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "c25c079023a4c6f7643f1b1151f1152b"
server_key_iv: "d33d30c590052e70ac617913546d68e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.083204 26406 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 22:39:27.085628 26422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.086758 26406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:27.086923 26406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "c25c079023a4c6f7643f1b1151f1152b"
server_key_iv: "d33d30c590052e70ac617913546d68e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.087031 26406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:27.111295 26406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:27.114040 26406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:27.114239 26406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:27.121508 26406 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:41691
I20260810 22:39:27.121532 26477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:41691 every 8 connection(s)
I20260810 22:39:27.122421 26406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:27.124172 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26406
I20260810 22:39:27.124317 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:27.124593 20265 external_mini_cluster.cc:1428] Setting key e8762dba098eecdd4e15313b7bdb3f01
I20260810 22:39:27.125344 26478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:27.130812 26478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f: Bootstrap starting.
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:27.133163 26478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:27.134034 26478 log.cc:824] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:27.136318 26478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f: No bootstrap required, opened a new log
I20260810 22:39:27.138559 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.126013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.19.202.126:41691)
0810 22:39:27.126430 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.126445 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.126480 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:39:27.127079 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.127094 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.127394 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.127707 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.128508 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.129556 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.130382 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.130824 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.130992 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.133337 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:27.133348 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:27.133360 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:27.133385 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:27.135402 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.135828 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.135843 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.135849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.136006 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.136268 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.136271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.136272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.136828 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:27.137018 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:27.137281 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 22:39:27.137573 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":135,"threads_started":1}
I20260810 22:39:27.140750 26478 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41691 } }
I20260810 22:39:27.141019 26478 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:27.141079 26478 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfdf1a76bb6a40ff9bb83e9d4f73572f, State: Initialized, Role: FOLLOWER
I20260810 22:39:27.141438 26478 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41691 } }
I20260810 22:39:27.141551 26478 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:27.141595 26478 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:27.141662 26478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:27.142464 26478 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41691 } }
I20260810 22:39:27.142745 26478 leader_election.cc:302] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfdf1a76bb6a40ff9bb83e9d4f73572f; no voters:
I20260810 22:39:27.142987 26478 leader_election.cc:288] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:27.143116 26484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:27.143378 26484 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [term 1 LEADER]: Becoming Leader. State: Replica: cfdf1a76bb6a40ff9bb83e9d4f73572f, State: Running, Role: LEADER
I20260810 22:39:27.143734 26484 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41691 } }
I20260810 22:39:27.144215 26478 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:27.145319 26485 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41691 } } }
I20260810 22:39:27.145445 26485 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:27.145746 26486 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfdf1a76bb6a40ff9bb83e9d4f73572f. Latest consensus state: current_term: 1 leader_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdf1a76bb6a40ff9bb83e9d4f73572f" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41691 } } }
I20260810 22:39:27.145918 26486 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:27.145836 26493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:27.148602 26493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:27.153601 26493 catalog_manager.cc:1347] Generated new cluster ID: 946cd6e31c834e6aa89c9eb3e26d4d03
I20260810 22:39:27.153676 26493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:27.165274 26493 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:27.166373 26493 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:27.180745 26493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfdf1a76bb6a40ff9bb83e9d4f73572f: Generated new TSK 0
I20260810 22:39:27.181481 26493 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:27.212208 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41691
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:27.321949 26507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:27.322176 26507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:27.322216 26507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:27.326047 26507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:27.326093 26507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:27.326148 26507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:27.331027 26507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:27.331949 26507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:27.332886 26507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:27.342326 26513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:27.342314 26512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:27.342401 26507 server_base.cc:1034] running on GCE node
W20260810 22:39:27.342314 26515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:27.342934 26507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:27.343583 26507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:27.344751 26507 hybrid_clock.cc:648] HybridClock initialized: now 1786401567344737 us; error 23 us; skew 500 ppm
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.347474 26507 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:27.348276 26507 webserver.cc:460] Webserver started at http://127.19.202.65:40909/ using document root <none> and password file <none>
I20260810 22:39:27.348888 26507 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:27.348980 26507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:27.349201 26507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:27.351110 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "7ee693b3d34743379c2218890446573e"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "a47bdd2d4df9b17f69c16c57b3bc3fcf"
server_key_iv: "1c37529795ffc19037640ab3e39be2e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.351655 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "7ee693b3d34743379c2218890446573e"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "a47bdd2d4df9b17f69c16c57b3bc3fcf"
server_key_iv: "1c37529795ffc19037640ab3e39be2e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.355201 26507 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:39:27.357595 26522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.358752 26507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:27.358889 26507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "7ee693b3d34743379c2218890446573e"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "a47bdd2d4df9b17f69c16c57b3bc3fcf"
server_key_iv: "1c37529795ffc19037640ab3e39be2e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.358997 26507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:27.371165 26507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:27.373988 26507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:27.374193 26507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:27.374761 26507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:27.375553 26507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:27.375624 26507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.375687 26507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:27.375730 26507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.384636 26507 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:32873
I20260810 22:39:27.384657 26645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:32873 every 8 connection(s)
I20260810 22:39:27.385578 26507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:27.388449 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26507
I20260810 22:39:27.388556 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:27.388880 20265 external_mini_cluster.cc:1428] Setting key 8e51f70767d39b5543eb467d999615e5
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:39:27.399928 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.387389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:60319 (local address 127.19.202.126:41691)
0810 22:39:27.387529 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.387533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.388366 (+ 833us) server_negotiation.cc:407] Connection header received
0810 22:39:27.389985 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.389989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.390040 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.390133 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.391596 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.392139 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.393338 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.393588 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.393703 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.396241 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:27.396246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:27.396253 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:27.396275 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:27.397826 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.398333 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.398337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.398338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.398426 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.398757 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.398761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.398763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.398978 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:27.399081 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:27.399554 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 22:39:27.399679 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:39:27.400632 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.387779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:41691 (local address 127.19.202.65:60319)
0810 22:39:27.388226 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:27.388266 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:27.389770 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:27.390306 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:27.390313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:27.390691 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:27.391451 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:27.391467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.392266 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.392269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:27.393185 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:27.393196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.393744 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.393748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:27.393928 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.394512 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:27.394527 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:27.396086 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:27.397965 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:27.397970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:27.397983 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:27.398229 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:27.398526 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:27.398529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:27.398531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:27.398662 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:27.399082 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:27.399089 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:27.399319 (+ 230us) client_negotiation.cc:769] Sending connection context
0810 22:39:27.399500 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 22:39:27.399835 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:39:27.401942 26646 heartbeater.cc:347] Connected to a master server at 127.19.202.126:41691
I20260810 22:39:27.402191 26646 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:27.402666 26646 heartbeater.cc:511] Master 127.19.202.126:41691 requested a full tablet report, sending...
I20260810 22:39:27.404134 26441 ts_manager.cc:194] Registered new tserver with Master: 7ee693b3d34743379c2218890446573e (127.19.202.65:32873)
I20260810 22:39:27.405395 26441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:60319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:27.416088 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41691
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:27.521446 26651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:27.521672 26651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:27.521741 26651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:27.525480 26651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:27.525561 26651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:27.525663 26651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:27.530330 26651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:27.531183 26651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:27.532100 26651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:27.541182 26656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:27.541177 26657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:27.541330 26651 server_base.cc:1034] running on GCE node
W20260810 22:39:27.541227 26659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:27.541801 26651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:27.542429 26651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:27.543619 26651 hybrid_clock.cc:648] HybridClock initialized: now 1786401567543602 us; error 28 us; skew 500 ppm
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.546423 26651 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:27.547225 26651 webserver.cc:460] Webserver started at http://127.19.202.66:33541/ using document root <none> and password file <none>
I20260810 22:39:27.547742 26651 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:27.547823 26651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:27.548038 26651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:27.549932 26651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "99d3ccba736249458c5e8d5e93b70d51"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "63404a06fcb282ba5b5a8049ca7abb22"
server_key_iv: "cd8e21874ea35457a1070cbf7c727c83"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.550441 26651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "99d3ccba736249458c5e8d5e93b70d51"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "63404a06fcb282ba5b5a8049ca7abb22"
server_key_iv: "cd8e21874ea35457a1070cbf7c727c83"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.553570 26651 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:39:27.555931 26667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.557070 26651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:27.557205 26651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "99d3ccba736249458c5e8d5e93b70d51"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "63404a06fcb282ba5b5a8049ca7abb22"
server_key_iv: "cd8e21874ea35457a1070cbf7c727c83"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.557317 26651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:27.574036 26651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:27.577214 26651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:27.577404 26651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:27.577975 26651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:27.578707 26651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:27.578773 26651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.578831 26651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:27.578877 26651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.587484 26651 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:41411
I20260810 22:39:27.587508 26788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:41411 every 8 connection(s)
I20260810 22:39:27.588272 26651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:27.591706 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26651
I20260810 22:39:27.591807 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:27.592024 20265 external_mini_cluster.cc:1428] Setting key 496a602cd698a8907170aa63e0509108
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:39:27.603183 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.589883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:47839 (local address 127.19.202.126:41691)
0810 22:39:27.590027 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.590031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.590803 (+ 772us) server_negotiation.cc:407] Connection header received
0810 22:39:27.593311 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.593315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.593364 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.593459 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.594987 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.595474 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.596588 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.596937 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.597021 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.599666 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:27.599669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:27.599671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:27.599693 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:27.601140 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.601743 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.601745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.601746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.601817 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.602182 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.602184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.602185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.602333 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:27.602404 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:27.602990 (+ 586us) server_negotiation.cc:299] Negotiation successful
0810 22:39:27.603078 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:39:27.604004 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.590234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:41691 (local address 127.19.202.66:47839)
0810 22:39:27.590673 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:27.590713 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:27.593082 (+ 2369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:27.593665 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:27.593673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:27.594160 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:27.594842 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:27.594864 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.595614 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.595620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:27.596459 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:27.596468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.597066 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.597069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:27.597214 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.597756 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:27.597772 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:27.599498 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:27.601260 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:27.601267 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:27.601281 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:27.601624 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:27.601908 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:27.601911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:27.601920 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:27.602090 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:27.602447 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:27.602455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:27.602718 (+ 263us) client_negotiation.cc:769] Sending connection context
0810 22:39:27.602908 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 22:39:27.603239 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":159,"threads_started":1}
I20260810 22:39:27.605131 26789 heartbeater.cc:347] Connected to a master server at 127.19.202.126:41691
I20260810 22:39:27.605345 26789 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
I20260810 22:39:27.605860 26789 heartbeater.cc:511] Master 127.19.202.126:41691 requested a full tablet report, sending...
I20260810 22:39:27.606961 26441 ts_manager.cc:194] Registered new tserver with Master: 99d3ccba736249458c5e8d5e93b70d51 (127.19.202.66:41411)
I20260810 22:39:27.607517 26441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:47839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:27.619922 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41691
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:27.726613 26795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:27.726825 26795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:27.726866 26795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:27.730576 26795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:27.730618 26795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:27.730713 26795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:27.735517 26795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:27.736423 26795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:27.737392 26795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:27.746337 26803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:27.746347 26801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:27.746347 26800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:27.746814 26795 server_base.cc:1034] running on GCE node
I20260810 22:39:27.747110 26795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:27.747663 26795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:27.748839 26795 hybrid_clock.cc:648] HybridClock initialized: now 1786401567748807 us; error 41 us; skew 500 ppm
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.751374 26795 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:27.752132 26795 webserver.cc:460] Webserver started at http://127.19.202.67:44741/ using document root <none> and password file <none>
I20260810 22:39:27.752584 26795 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:27.752655 26795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:27.752864 26795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:27.754690 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "5ae6bc0022634027be886b5859cf4229"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "ec4590d7bb0b8e2ec761cf5b02fbd31e"
server_key_iv: "3b294bba5bef9912eed86c3304ac631b"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.755174 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "5ae6bc0022634027be886b5859cf4229"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "ec4590d7bb0b8e2ec761cf5b02fbd31e"
server_key_iv: "3b294bba5bef9912eed86c3304ac631b"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.758672 26795 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 22:39:27.760999 26810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.762229 26795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:27.762362 26795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "5ae6bc0022634027be886b5859cf4229"
format_stamp: "Formatted at 2026-08-10 22:39:27 on dist-test-slave-1x4g"
server_key: "ec4590d7bb0b8e2ec761cf5b02fbd31e"
server_key_iv: "3b294bba5bef9912eed86c3304ac631b"
server_key_version: "encryptionkey@0"
I20260810 22:39:27.762485 26795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:27.773203 26795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:27.777063 26795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:27.777257 26795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:27.777881 26795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:27.778770 26795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:27.778848 26795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.778919 26795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:27.778954 26795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:27.788100 26795 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:40831
I20260810 22:39:27.788154 26935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:40831 every 8 connection(s)
I20260810 22:39:27.789077 26795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:27.795248 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26795
I20260810 22:39:27.795365 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:27.795665 20265 external_mini_cluster.cc:1428] Setting key c66fbafd9121a404ed4be57128d1f934
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:27.802855 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.791038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:42275 (local address 127.19.202.126:41691)
0810 22:39:27.791168 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.791189 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.792035 (+ 846us) server_negotiation.cc:407] Connection header received
0810 22:39:27.793455 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.793458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.793506 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.793586 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.794703 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.795222 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.796857 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.797095 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.797204 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.799606 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:27.799616 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:27.799619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:27.799641 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:27.800897 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.801391 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.801395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.801397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.801479 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.801820 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.801824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.801825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.802002 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:27.802108 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:27.802575 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 22:39:27.802695 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:27.803622 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.791430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:41691 (local address 127.19.202.67:42275)
0810 22:39:27.791892 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:27.791938 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:27.793251 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:27.793741 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:27.793748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:27.794108 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:27.794574 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:27.794587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.795356 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.795359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:27.796702 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:27.796713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.797242 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.797245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:27.797389 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.798029 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:27.798045 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:27.799449 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:27.801007 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:27.801013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:27.801026 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:27.801277 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:27.801571 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:27.801575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:27.801577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:27.801715 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:27.802108 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:27.802115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:27.802348 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 22:39:27.802535 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 22:39:27.802832 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
I20260810 22:39:27.804672 26936 heartbeater.cc:347] Connected to a master server at 127.19.202.126:41691
I20260810 22:39:27.804951 26936 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:27.805398 26936 heartbeater.cc:511] Master 127.19.202.126:41691 requested a full tablet report, sending...
I20260810 22:39:27.806401 26441 ts_manager.cc:194] Registered new tserver with Master: 5ae6bc0022634027be886b5859cf4229 (127.19.202.67:40831)
I20260810 22:39:27.806978 26441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:42275
I20260810 22:39:27.809571 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/test-admin.keytab.
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.824831 20265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 22:39:27.827749 26944 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:27.826231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.19.202.65:32873)
0810 22:39:27.826594 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.826604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.826625 (+ 21us) server_negotiation.cc:407] Connection header received
0810 22:39:27.826672 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.826677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.826899 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.827020 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.827561 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37874: BlockingRecv error: recv got EOF from 127.0.0.1:37874 (error 108)
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":160,"threads_started":1}
W20260810 22:39:27.827762 20265 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc' of type FILE
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.828691 20265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.829600 20265 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:27.838414 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.830397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.19.202.65:32873)
0810 22:39:27.830543 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.830546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.830598 (+ 52us) server_negotiation.cc:407] Connection header received
0810 22:39:27.830690 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.830692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.830737 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.830796 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.831288 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.832027 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.832832 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.833183 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.833288 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.835407 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:27.835416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:27.835428 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:27.835449 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:27.836906 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.837293 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.837298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.837299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.837387 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.837614 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.837616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.837617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.837859 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:27.838014 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:27.838172 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 22:39:27.838287 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 22:39:27.841763 26944 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:27.840987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37896 (local address 127.19.202.65:32873)
0810 22:39:27.841085 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.841089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.841202 (+ 113us) server_negotiation.cc:407] Connection header received
0810 22:39:27.841273 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.841275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.841305 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.841365 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.841674 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37896: BlockingRecv error: recv got EOF from 127.0.0.1:37896 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 22:39:27.842046 20265 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc)
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.843006 20265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.843875 20265 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:27.851429 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.844639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.19.202.65:32873)
0810 22:39:27.844767 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.844814 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.845035 (+ 221us) server_negotiation.cc:407] Connection header received
0810 22:39:27.845103 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.845105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.845136 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.845200 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.845681 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.846227 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.847012 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.847215 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.847290 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.849155 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:27.849157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:27.849159 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:27.849178 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:27.850251 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.850586 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.850588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.850589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.850654 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.850856 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.850858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.850859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.851004 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:27.851087 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:27.851260 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 22:39:27.851341 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 22:39:27.853561 26944 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:27.852903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37910 (local address 127.19.202.65:32873)
0810 22:39:27.853011 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.853014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.853037 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:39:27.853118 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.853120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.853146 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.853201 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.853472 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37910: BlockingRecv error: recv got EOF from 127.0.0.1:37910 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 22:39:27.853688 20265 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786401536977653-20265-0/krb5kdc/krb5cc' of type FILE
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.854576 20265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:27.855392 20265 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:39:27.863154 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:27.856086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.19.202.65:32873)
0810 22:39:27.856209 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:27.856213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:27.856268 (+ 55us) server_negotiation.cc:407] Connection header received
0810 22:39:27.856346 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:27.856348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:27.856374 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:27.856433 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:27.856939 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.857403 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.858122 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:27.858296 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:27.858362 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:27.860386 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:27.860388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:27.860390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:27.860409 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:27.861729 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.862138 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.862141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.862142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.862243 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:27.862501 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:27.862504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:27.862505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:27.862658 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:27.862744 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:27.862893 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 22:39:27.862995 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 22:39:27.864180 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26507
I20260810 22:39:27.873821 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26651
I20260810 22:39:27.881888 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26795
I20260810 22:39:27.888491 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26406
2026-08-10T22:39:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2944 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26958](info): set up 2 sockets
Aug 10 22:39:27 dist-test-slave-1x4g krb5kdc[26958](info): commencing operation
krb5kdc: starting...
W20260810 22:39:28.954656 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab.
Entry for principal oryx/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab.
I20260810 22:39:29.075485 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34879
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab
--principal=oryx/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:29.183609 26971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:29.183890 26971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:29.183959 26971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:29.187795 26971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:29.187877 26971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:29.187913 26971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:29.187950 26971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:29.187983 26971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:29.192842 26971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab
--principal=oryx/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:29.193764 26971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:29.194727 26971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:29.202764 26976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:29.202764 26979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:29.202811 26977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:29.203073 26971 server_base.cc:1034] running on GCE node
I20260810 22:39:29.203567 26971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:29.204553 26971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:29.205740 26971 hybrid_clock.cc:648] HybridClock initialized: now 1786401569205714 us; error 36 us; skew 500 ppm
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:29.208359 26971 init.cc:377] Logged in from keytab as oryx/127.19.202.126@KRBTEST.COM (short username oryx)
I20260810 22:39:29.209261 26971 webserver.cc:460] Webserver started at http://127.19.202.126:40299/ using document root <none> and password file <none>
I20260810 22:39:29.209791 26971 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:29.209851 26971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:29.210115 26971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:29.212002 26971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "fcd41744f48a4235ae0ca99f5a08bd61"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "7e5d2d10ea63e03166256f8c59ba85d0"
server_key_iv: "889269370343f55808c125fd48fa5b52"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.212555 26971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "fcd41744f48a4235ae0ca99f5a08bd61"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "7e5d2d10ea63e03166256f8c59ba85d0"
server_key_iv: "889269370343f55808c125fd48fa5b52"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.216081 26971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:29.218392 26986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.219486 26971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:29.219621 26971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "fcd41744f48a4235ae0ca99f5a08bd61"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "7e5d2d10ea63e03166256f8c59ba85d0"
server_key_iv: "889269370343f55808c125fd48fa5b52"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.219718 26971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:29.229727 26971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:29.232532 26971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:29.232713 26971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:29.239806 26971 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:34879
I20260810 22:39:29.240065 27043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:34879 every 8 connection(s)
I20260810 22:39:29.240753 26971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:29.243432 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:29.248656 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: Bootstrap starting.
I20260810 22:39:29.251044 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 26971
I20260810 22:39:29.251197 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:29.251216 27044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:29.251555 20265 external_mini_cluster.cc:1428] Setting key 5477073ac049ca1b4c0f45a67390affa
I20260810 22:39:29.252091 27044 log.cc:824] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:29.254110 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: No bootstrap required, opened a new log
I20260810 22:39:29.257594 27044 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } }
I20260810 22:39:29.257826 27044 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:29.257929 27044 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcd41744f48a4235ae0ca99f5a08bd61, State: Initialized, Role: FOLLOWER
I20260810 22:39:29.258455 27044 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } }
I20260810 22:39:29.258630 27044 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:29.258703 27044 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:29.258801 27044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:29.259716 27044 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } }
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:29.260076 27044 leader_election.cc:302] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcd41744f48a4235ae0ca99f5a08bd61; no voters:
I20260810 22:39:29.260411 27044 leader_election.cc:288] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:29.260520 27049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:29.260797 27049 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 1 LEADER]: Becoming Leader. State: Replica: fcd41744f48a4235ae0ca99f5a08bd61, State: Running, Role: LEADER
I20260810 22:39:29.261191 27049 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } }
I20260810 22:39:29.261724 27044 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:29.262863 27051 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } }
I20260810 22:39:29.263062 27051 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:29.263376 27051 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcd41744f48a4235ae0ca99f5a08bd61. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } }
I20260810 22:39:29.263528 27051 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:29.266090 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:29.252664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52178 (local address 127.19.202.126:34879)
0810 22:39:29.253127 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:29.253143 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:29.253174 (+ 31us) server_negotiation.cc:407] Connection header received
0810 22:39:29.253767 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:29.253801 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:29.254102 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:29.254483 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:29.255265 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.256285 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.257223 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.257690 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.257865 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:29.260661 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:29.260667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:29.260685 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:29.260719 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:29.262851 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.263381 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.263387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.263392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.263524 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.263845 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.263849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.263851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.264369 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:29.264555 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:29.264881 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 22:39:29.265204 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":178,"threads_started":1}
W20260810 22:39:29.266618 27067 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:39:29.266717 27067 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:39:29.266831 27062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:29.267678 27062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:29.273051 27062 catalog_manager.cc:1347] Generated new cluster ID: 3ccf87bfb5c24f359435f4cd4289174a
I20260810 22:39:29.273116 27062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:29.281630 27062 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:29.282658 27062 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:29.286835 27062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: Generated new TSK 0
I20260810 22:39:29.287386 27062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab.
Entry for principal oryx/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab.
I20260810 22:39:29.319221 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34879
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab
--principal=oryx/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:29.443041 27073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:29.443259 27073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:29.443298 27073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:29.447269 27073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:29.447320 27073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:29.447376 27073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:29.452235 27073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab
--principal=oryx/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:29.453154 27073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:29.454092 27073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:29.463467 27079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:29.463534 27078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:29.463481 27081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:29.464138 27073 server_base.cc:1034] running on GCE node
I20260810 22:39:29.464511 27073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:29.465176 27073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:29.466352 27073 hybrid_clock.cc:648] HybridClock initialized: now 1786401569466320 us; error 45 us; skew 500 ppm
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:29.468997 27073 init.cc:377] Logged in from keytab as oryx/127.19.202.65@KRBTEST.COM (short username oryx)
I20260810 22:39:29.469852 27073 webserver.cc:460] Webserver started at http://127.19.202.65:38577/ using document root <none> and password file <none>
I20260810 22:39:29.470417 27073 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:29.470506 27073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:29.470736 27073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:29.472697 27073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "8d0cb262e1124afa902aa295415cc336"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "934cf9bf4bda22890d1b0cd603a265d0"
server_key_iv: "e7b42e304d330f04c55dbe9ba78e7934"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.473301 27073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "8d0cb262e1124afa902aa295415cc336"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "934cf9bf4bda22890d1b0cd603a265d0"
server_key_iv: "e7b42e304d330f04c55dbe9ba78e7934"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.476967 27073 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:29.479326 27088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.480522 27073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:29.480667 27073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "8d0cb262e1124afa902aa295415cc336"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "934cf9bf4bda22890d1b0cd603a265d0"
server_key_iv: "e7b42e304d330f04c55dbe9ba78e7934"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.480782 27073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:29.499063 27073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:29.501688 27073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:29.501881 27073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:29.502427 27073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:29.503197 27073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:29.503266 27073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.503335 27073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:29.503377 27073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.512383 27073 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:44641
I20260810 22:39:29.512410 27213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:44641 every 8 connection(s)
I20260810 22:39:29.513237 27073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:29.516494 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 27073
I20260810 22:39:29.516637 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:29.516983 20265 external_mini_cluster.cc:1428] Setting key b966d39561f008a3273126fc29884ffa
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:29.527190 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:29.514953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:34071 (local address 127.19.202.126:34879)
0810 22:39:29.515090 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:29.515093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:29.515873 (+ 780us) server_negotiation.cc:407] Connection header received
0810 22:39:29.517454 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:29.517457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:29.517516 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:29.517632 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:29.518767 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.519314 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.520379 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.520609 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.520712 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:29.523193 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:29.523197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:29.523200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:29.523230 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:29.525041 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.525572 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.525579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.525581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.525678 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.526013 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.526017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.526020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.526258 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:29.526369 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:29.526847 (+ 478us) server_negotiation.cc:299] Negotiation successful
0810 22:39:29.527008 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for oryx/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:39:29.527858 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:29.515283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:34071)
0810 22:39:29.515732 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:29.515771 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:29.517235 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:29.517808 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:29.517814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:29.518190 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:29.518628 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:29.518640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.519453 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.519456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:29.520258 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:29.520266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.520747 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.520751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:29.520960 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:29.521563 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:29.521578 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:29.523045 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:29.525183 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:29.525191 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:29.525208 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:29.525466 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:29.525771 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:29.525774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:29.525775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:29.525917 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:29.526362 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:29.526366 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:29.526591 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 22:39:29.526732 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 22:39:29.526997 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":123,"threads_started":1}
I20260810 22:39:29.529105 27214 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34879
I20260810 22:39:29.529358 27214 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:29.529953 27214 heartbeater.cc:511] Master 127.19.202.126:34879 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.66@KRBTEST.COM" created.
I20260810 22:39:29.531618 27005 ts_manager.cc:194] Registered new tserver with Master: 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641)
I20260810 22:39:29.532696 27005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:34071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab.
Entry for principal oryx/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab.
I20260810 22:39:29.545725 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34879
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab
--principal=oryx/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:29.652657 27219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:29.652969 27219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:29.653050 27219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:29.656971 27219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:29.657050 27219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:29.657135 27219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:29.662391 27219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab
--principal=oryx/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:29.663424 27219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:29.664405 27219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:29.673739 27225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:29.673738 27227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:29.673740 27224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:29.673997 27219 server_base.cc:1034] running on GCE node
I20260810 22:39:29.674535 27219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:29.675072 27219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:29.676216 27219 hybrid_clock.cc:648] HybridClock initialized: now 1786401569676190 us; error 39 us; skew 500 ppm
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:29.678857 27219 init.cc:377] Logged in from keytab as oryx/127.19.202.66@KRBTEST.COM (short username oryx)
I20260810 22:39:29.679656 27219 webserver.cc:460] Webserver started at http://127.19.202.66:41067/ using document root <none> and password file <none>
I20260810 22:39:29.680197 27219 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:29.680284 27219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:29.680514 27219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:29.682546 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "be93acea2f3f4a8caad402f6374d483c"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "7ab76306532591dc18c057ea90bf9f60"
server_key_iv: "1d5678a5cf7efd3d40689c1a8269d9ca"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.683066 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "be93acea2f3f4a8caad402f6374d483c"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "7ab76306532591dc18c057ea90bf9f60"
server_key_iv: "1d5678a5cf7efd3d40689c1a8269d9ca"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.686779 27219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:39:29.689014 27234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.690160 27219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:29.690286 27219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "be93acea2f3f4a8caad402f6374d483c"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "7ab76306532591dc18c057ea90bf9f60"
server_key_iv: "1d5678a5cf7efd3d40689c1a8269d9ca"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.690388 27219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:29.711613 27219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:29.714355 27219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:29.714560 27219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:29.715134 27219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:29.715879 27219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:29.715952 27219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.716022 27219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:29.716060 27219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.725351 27219 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:40455
I20260810 22:39:29.725373 27354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:40455 every 8 connection(s)
I20260810 22:39:29.726140 27219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:29.731456 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 27219
I20260810 22:39:29.731544 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:29.731815 20265 external_mini_cluster.cc:1428] Setting key 509d492c790fbbf632ea7dc0ba95b54a
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:29.739684 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:29.727778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:34937 (local address 127.19.202.126:34879)
0810 22:39:29.727919 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:29.727923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:29.728697 (+ 774us) server_negotiation.cc:407] Connection header received
0810 22:39:29.729952 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:29.729955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:29.729998 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:29.730066 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:29.731102 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.731672 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.732891 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.733214 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.733345 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:29.735971 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:29.735974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:29.735976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:29.736017 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:29.737435 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.738019 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.738026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.738029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.738130 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.738465 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.738468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.738470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.738729 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:29.738847 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:29.739319 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 22:39:29.739469 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:39:29.740278 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:29.728140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:34937)
0810 22:39:29.728559 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:29.728598 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:29.729769 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:29.730219 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:29.730225 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:29.730553 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:29.730987 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:29.730999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.731794 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.731796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:29.732737 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:29.732747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.733393 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.733397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:29.733565 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:29.734140 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:29.734156 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:29.735800 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:29.737556 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:29.737563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:29.737577 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:29.737853 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:29.738232 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:29.738234 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:29.738236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:29.738363 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:29.738836 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:29.738841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:29.739071 (+ 230us) client_negotiation.cc:769] Sending connection context
0810 22:39:29.739227 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 22:39:29.739448 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260810 22:39:29.741598 27355 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34879
I20260810 22:39:29.741835 27355 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:39:29.742324 27355 heartbeater.cc:511] Master 127.19.202.126:34879 requested a full tablet report, sending...
I20260810 22:39:29.743495 27005 ts_manager.cc:194] Registered new tserver with Master: be93acea2f3f4a8caad402f6374d483c (127.19.202.66:40455)
I20260810 22:39:29.744113 27005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:34937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab.
Entry for principal oryx/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab.
I20260810 22:39:29.759946 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34879
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab
--principal=oryx/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:29.868569 27360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:29.868855 27360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:29.868991 27360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:29.872731 27360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:29.872821 27360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:29.872924 27360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:29.877718 27360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab
--principal=oryx/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:29.878604 27360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:29.879472 27360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:29.888428 27367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:29.888550 27360 server_base.cc:1034] running on GCE node
W20260810 22:39:29.888428 27369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:29.888480 27366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:29.889153 27360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:29.889745 27360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:29.890938 27360 hybrid_clock.cc:648] HybridClock initialized: now 1786401569890916 us; error 46 us; skew 500 ppm
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:29.893507 27360 init.cc:377] Logged in from keytab as oryx/127.19.202.67@KRBTEST.COM (short username oryx)
I20260810 22:39:29.894379 27360 webserver.cc:460] Webserver started at http://127.19.202.67:42277/ using document root <none> and password file <none>
I20260810 22:39:29.894860 27360 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:29.894917 27360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:29.895087 27360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:29.897150 27360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "6361f8d43c044c939eaa4585da651f4d"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "768b8b75da6ebb23f013a04f84436959"
server_key_iv: "a94817ec1daeb61f8cec379398407c80"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.897675 27360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "6361f8d43c044c939eaa4585da651f4d"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "768b8b75da6ebb23f013a04f84436959"
server_key_iv: "a94817ec1daeb61f8cec379398407c80"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.901013 27360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:39:29.903285 27376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.904439 27360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:29.904539 27360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "6361f8d43c044c939eaa4585da651f4d"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "768b8b75da6ebb23f013a04f84436959"
server_key_iv: "a94817ec1daeb61f8cec379398407c80"
server_key_version: "encryptionkey@0"
I20260810 22:39:29.904619 27360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:29.930655 27360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:29.933279 27360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:29.933416 27360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:29.933911 27360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:29.934620 27360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:29.934665 27360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.934702 27360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:29.934718 27360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:29.943327 27360 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:39199
I20260810 22:39:29.943372 27498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:39199 every 8 connection(s)
I20260810 22:39:29.944190 27360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:29.946219 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 27360
I20260810 22:39:29.946336 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:29.946599 20265 external_mini_cluster.cc:1428] Setting key 5ca1a15ff0449109da398a65ae694373
Aug 10 22:39:29 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:29.956481 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:29.945820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:43433 (local address 127.19.202.126:34879)
0810 22:39:29.945965 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:29.945968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:29.946705 (+ 737us) server_negotiation.cc:407] Connection header received
0810 22:39:29.948327 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:29.948330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:29.948378 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:29.948459 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:29.949567 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.950099 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.951152 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.951352 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.951424 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:29.953603 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:29.953606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:29.953608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:29.953629 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:29.954845 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.955279 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.955281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.955282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.955350 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:29.955626 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:29.955628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:29.955629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:29.955784 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:29.955856 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:29.956262 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 22:39:29.956350 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:29.957273 27502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:29.946114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:43433)
0810 22:39:29.946568 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:29.946614 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:29.948145 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:29.948660 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:29.948666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:29.949015 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:29.949443 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:29.949456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.950205 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.950207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:29.951050 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:29.951059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:29.951468 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:29.951471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:29.951614 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:29.952058 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:29.952072 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:29.953462 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:29.954944 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:29.954949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:29.954973 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:29.955197 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:29.955425 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:29.955427 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:29.955428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:29.955547 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:29.955876 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:29.955881 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:29.956073 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 22:39:29.956215 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 22:39:29.956447 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260810 22:39:29.958330 27500 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34879
I20260810 22:39:29.958561 27500 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:29.959048 27500 heartbeater.cc:511] Master 127.19.202.126:34879 requested a full tablet report, sending...
I20260810 22:39:29.960040 27005 ts_manager.cc:194] Registered new tserver with Master: 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199)
I20260810 22:39:29.960611 27005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:43433
I20260810 22:39:29.971776 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.202.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.125.keytab.
Entry for principal oryx/127.19.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.125.keytab.
I20260810 22:39:29.997893 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:42995
--webserver_interface=127.19.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.125.keytab
--principal=oryx/127.19.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34879,127.19.202.125:42995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:30.107506 27505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:30.107725 27505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:30.107791 27505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:30.111685 27505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:30.111732 27505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:30.111750 27505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:30.111790 27505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:30.111821 27505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:30.116868 27505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34879,127.19.202.125:42995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.125.keytab
--principal=oryx/127.19.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
W20260810 22:39:30.117053 27505 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.202.126:34879,127.19.202.125:42995'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 22:39:30.117838 27505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:30.118767 27505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:30.126919 27513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.126956 27511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.126936 27510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:30.127233 27505 server_base.cc:1034] running on GCE node
I20260810 22:39:30.127774 27505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:30.128748 27505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:30.129979 27505 hybrid_clock.cc:648] HybridClock initialized: now 1786401570129972 us; error 29 us; skew 500 ppm
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:30.132581 27505 init.cc:377] Logged in from keytab as oryx/127.19.202.125@KRBTEST.COM (short username oryx)
I20260810 22:39:30.133404 27505 webserver.cc:460] Webserver started at http://127.19.202.125:41793/ using document root <none> and password file <none>
I20260810 22:39:30.133908 27505 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.133965 27505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:30.134120 27505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:30.135983 27505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/instance:
uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "5b658c21d7a720451f2523698c3f04ee"
server_key_iv: "8c89ba4d631c7175b040674ff29f0dab"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.136487 27505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal/instance:
uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "5b658c21d7a720451f2523698c3f04ee"
server_key_iv: "8c89ba4d631c7175b040674ff29f0dab"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.139936 27505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 22:39:30.142243 27520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.143318 27505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 22:39:30.143445 27505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "5b658c21d7a720451f2523698c3f04ee"
server_key_iv: "8c89ba4d631c7175b040674ff29f0dab"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.143538 27505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:30.161167 27505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:30.163928 27505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:30.164103 27505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:30.170815 27505 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.125:42995
I20260810 22:39:30.171121 27577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.125:42995 every 8 connection(s)
I20260810 22:39:30.171715 27505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
I20260810 22:39:30.173657 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 27505
I20260810 22:39:30.174574 27578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:30.178722 27578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.202.125@KRBTEST.COM
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.125@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:30.187994 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.175121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.19.202.125:42995)
0810 22:39:30.175510 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.175524 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.175556 (+ 32us) server_negotiation.cc:407] Connection header received
0810 22:39:30.176144 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.176159 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.176514 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.176846 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.177713 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.178638 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.179714 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.180166 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.180278 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.182875 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.182880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.182889 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.182920 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.184927 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.185436 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.185441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.185445 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.185562 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.185878 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.185880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.185881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.186305 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.186498 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.186773 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.187049 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":138,"threads_started":1}
I20260810 22:39:30.189975 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.180532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.19.202.126:34879)
0810 22:39:30.180670 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.180673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.181194 (+ 521us) server_negotiation.cc:407] Connection header received
0810 22:39:30.181334 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.181337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.181394 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.181479 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.182093 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.182631 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.183689 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.183923 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.184042 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.186524 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.186528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.186530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.186554 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.188096 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.188613 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.188619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.188623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.188710 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.189046 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.189050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.189051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.189261 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.189370 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.189707 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.189804 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 22:39:30.189987 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.180468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:38958)
0810 22:39:30.181071 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.181106 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.181240 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.181528 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.181534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.181863 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.181976 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.181981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.182770 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.182773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.183568 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.183578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.184068 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.184071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.184192 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.184705 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.184716 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.186348 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.188217 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.188224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.188228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.188507 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.188823 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.188826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.188827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.188920 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.189376 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.189384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.189493 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.189626 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.189801 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":117,"threads_started":1}
I20260810 22:39:30.191169 27578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.125@KRBTEST.COM for oryx/127.19.202.125@KRBTEST.COM
I20260810 22:39:30.198882 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.191601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59342)
0810 22:39:30.191698 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.191714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.191798 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.192005 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.192008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.192157 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.192250 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.192254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.193348 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.193351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.193924 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.193934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.194357 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.194360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.194453 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.194804 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.194814 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.195929 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.197467 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.197473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.197476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.197658 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.197935 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.197938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.197940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.198015 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.198376 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.198379 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.198469 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.198566 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.198700 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 22:39:30.198890 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.191690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.19.202.125:42995)
0810 22:39:30.191785 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.191788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.191800 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:39:30.191874 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.191876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.191906 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.191994 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.192358 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.193225 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.194034 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.194253 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.194357 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.196040 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.196043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.196045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.196072 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.197362 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.197752 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.197758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.197759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.197843 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.198116 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.198120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.198122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.198264 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.198383 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.198619 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.198747 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":23}
I20260810 22:39:30.201502 27578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Bootstrap starting.
I20260810 22:39:30.203593 27578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:30.204320 27578 log.cc:824] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:30.206090 27578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: No bootstrap required, opened a new log
I20260810 22:39:30.208909 27578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } }
I20260810 22:39:30.209086 27578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:30.209167 27578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc88aa2aeb94b9983677db2536a7b8d, State: Initialized, Role: FOLLOWER
I20260810 22:39:30.209612 27578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } }
I20260810 22:39:30.210095 27585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } } }
I20260810 22:39:30.210222 27585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:39:30.210809 27578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: configured and running, proceeding with master startup.
W20260810 22:39:30.215441 27598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:39:30.215528 27598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:39:30.217504 27505 master_runner.cc:428] Detected that this master 1dc88aa2aeb94b9983677db2536a7b8d is joining an existing cluster
I20260810 22:39:30.217569 27505 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.202.125:42995
I20260810 22:39:30.228533 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.219582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.19.202.125:42995)
0810 22:39:30.219783 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.219787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.220056 (+ 269us) server_negotiation.cc:407] Connection header received
0810 22:39:30.220151 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.220154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.220198 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.220272 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.221132 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.221900 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.222967 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.223318 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.223459 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.224950 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.224954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.224957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.224984 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.226705 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.227210 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.227213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.227215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.227304 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.227635 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.227638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.227640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.227854 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.227953 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.228208 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.228352 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":128,"spinlock_wait_cycles":1408}
I20260810 22:39:30.228719 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.219491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.19.202.126:34879)
0810 22:39:30.219614 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.219620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.219785 (+ 165us) server_negotiation.cc:407] Connection header received
0810 22:39:30.219973 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.219977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.220031 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.220085 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.221146 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.221653 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.222849 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.223167 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.223236 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.225356 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.225362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.225368 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.225398 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.226983 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.227460 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.227476 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.227477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.227572 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.227868 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.227871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.227873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.228073 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.228159 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.228470 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.228589 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 22:39:30.228538 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.219682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59348)
0810 22:39:30.219962 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.219974 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.220055 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.220297 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.220301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.220582 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.221008 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.221014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.222020 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.222024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.222825 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.222835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.223426 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.223429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.223529 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.224201 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.224221 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.224745 (+ 524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.226823 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.226829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.226834 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.227118 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.227406 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.227410 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.227412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.227511 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.227939 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.227941 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.228030 (+ 89us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.228140 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.228286 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":111,"spinlock_wait_cycles":1280,"thread_start_us":104,"threads_started":1}
I20260810 22:39:30.229256 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.219451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:38968)
0810 22:39:30.219703 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.219719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.219856 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.220239 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.220263 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.220585 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.221005 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.221014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.221764 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.221769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.222691 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.222703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.223393 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.223397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.223504 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.224813 (+ 1309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.224832 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.225187 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.227106 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.227110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.227111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.227326 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.227675 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.227678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.227679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.227772 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.228200 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.228203 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.228290 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.228950 (+ 660us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.229080 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":155,"spinlock_wait_cycles":4736,"thread_start_us":99,"threads_started":1}
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.126@KRBTEST.COM for oryx/127.19.202.125@KRBTEST.COM
I20260810 22:39:30.245563 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.233555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.19.202.125:42995)
0810 22:39:30.233668 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.233672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.234022 (+ 350us) server_negotiation.cc:407] Connection header received
0810 22:39:30.234204 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.234207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.234258 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.234341 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.234907 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.235641 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.236470 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.236872 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.236974 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.239263 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.239267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.239270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.239294 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.240954 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.244082 (+ 3128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.244085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.244087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.244171 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.244483 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.244486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.244487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.244732 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.244861 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.245296 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.245419 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:30.245582 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.233525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59354)
0810 22:39:30.233886 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.233946 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.234073 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.234377 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.234382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.234701 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.234798 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.234803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.235758 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.235761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.236365 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.236373 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.236978 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.236981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.237100 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.237615 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.237626 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.239113 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.243629 (+ 4516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.243640 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.243646 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.243956 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.244272 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.244275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.244277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.244378 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.244922 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.244928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.245058 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.245207 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.245414 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":119,"threads_started":1}
I20260810 22:39:30.247416 27005 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.202.125:42995: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
member_type: NON_VOTER
last_known_addr {
host: "127.19.202.125"
port: 42995
}
attrs {
promote: true
}
}
dest_uuid: "fcd41744f48a4235ae0ca99f5a08bd61"
cas_config_opid_index: -1
I20260810 22:39:30.248116 27005 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } }
I20260810 22:39:30.250635 27549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:30.252113 27549 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Refusing update from remote peer fcd41744f48a4235ae0ca99f5a08bd61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 22:39:30.252966 27608 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } }
WARNING: no policy specified for oryx/127.19.202.124@KRBTEST.COM; defaulting to no policy
I20260810 22:39:30.254225 27610 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:30.254822 27608 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.202.125:42995
I20260810 22:39:30.255357 27505 master.cc:537] Master@127.19.202.125:42995 shutting down...
I20260810 22:39:30.255585 27609 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } }
I20260810 22:39:30.255879 27609 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.124@KRBTEST.COM" created.
I20260810 22:39:30.261458 27505 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:30.261739 27505 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:30.261891 27505 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: stopping tablet replica
W20260810 22:39:30.265391 26992 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 22:39:30.266686 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.266333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59364)
0810 22:39:30.266514 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.266612 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260810 22:39:30.266930 26992 consensus_peers.cc:487] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 -> Peer 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125:42995): Couldn't send request to peer 1dc88aa2aeb94b9983677db2536a7b8d. Status: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42995: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 22:39:30.268047 27505 master.cc:552] Master@127.19.202.125:42995 shutdown complete.
I20260810 22:39:30.268087 27505 master_runner.cc:305] Clearing existing system tablet
I20260810 22:39:30.268270 27505 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.268374 27505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:30.269977 27505 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:39:30.270961 27617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.271276 27505 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:30.271387 27505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:30.271448 27505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "5b658c21d7a720451f2523698c3f04ee"
server_key_iv: "8c89ba4d631c7175b040674ff29f0dab"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.273388 27505 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.124.keytab.
Entry for principal oryx/127.19.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.124.keytab.
I20260810 22:39:30.275797 27505 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 22:39:30.275905 27505 log.cc:1196] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 22:39:30.275921 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:36949
--webserver_interface=127.19.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.124.keytab
--principal=oryx/127.19.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:39:30.276134 27505 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Deleting consensus metadata
I20260810 22:39:30.276324 27505 master_runner.cc:315] Copying system tablet from 127.19.202.126:34879
I20260810 22:39:30.277510 27505 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.202.126:34879
I20260810 22:39:30.286427 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.279225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.19.202.126:34879)
0810 22:39:30.279366 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.279370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.279504 (+ 134us) server_negotiation.cc:407] Connection header received
0810 22:39:30.279655 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.279657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.279702 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.279779 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.280206 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.280756 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.281601 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.281820 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.281926 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.282980 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.282984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.282986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.283008 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.284597 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.285218 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.285222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.285224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.285302 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.285617 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.285621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.285622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.285788 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.285888 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.286168 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.286276 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:39:30.286440 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.279118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:38978)
0810 22:39:30.279390 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.279403 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.279552 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.279795 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.279798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.279980 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.280094 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.280100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.280892 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.280897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.281478 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.281487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.281928 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.281931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.282024 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.282542 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.282556 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.282849 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.284700 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.284705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.284707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.285131 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.285427 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.285437 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.285438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.285512 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.285886 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.285889 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.286004 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.286111 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.286251 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
I20260810 22:39:30.287230 27028 tablet_copy_service.cc:140] P fcd41744f48a4235ae0ca99f5a08bd61: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1dc88aa2aeb94b9983677db2536a7b8d ({username='oryx', principal='oryx/127.19.202.125@KRBTEST.COM'} at 127.0.0.1:38978)
I20260810 22:39:30.287398 27028 tablet_copy_service.cc:161] P fcd41744f48a4235ae0ca99f5a08bd61: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1dc88aa2aeb94b9983677db2536a7b8d at {username='oryx', principal='oryx/127.19.202.125@KRBTEST.COM'} at 127.0.0.1:38978: session id = 1dc88aa2aeb94b9983677db2536a7b8d-00000000000000000000000000000000
I20260810 22:39:30.288767 27028 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 22:39:30.290280 27505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 22:39:30.292021 27505 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: tablet copy: Starting download of 0 data blocks...
I20260810 22:39:30.292224 27505 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: tablet copy: Starting download of 1 WAL segments...
I20260810 22:39:30.293787 27505 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 22:39:30.295356 27028 tablet_copy_service.cc:342] P fcd41744f48a4235ae0ca99f5a08bd61: Request end of tablet copy session 1dc88aa2aeb94b9983677db2536a7b8d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.202.125@KRBTEST.COM'} at 127.0.0.1:38978
I20260810 22:39:30.295462 27028 tablet_copy_service.cc:433] P fcd41744f48a4235ae0ca99f5a08bd61: ending tablet copy session 1dc88aa2aeb94b9983677db2536a7b8d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1dc88aa2aeb94b9983677db2536a7b8d
I20260810 22:39:30.301702 27505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:30.303165 27628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.303326 27625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.303449 27626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:30.303758 27505 server_base.cc:1034] running on GCE node
I20260810 22:39:30.303974 27505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:30.304215 27505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:30.305339 27505 hybrid_clock.cc:648] HybridClock initialized: now 1786401570305328 us; error 27 us; skew 500 ppm
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:30.306705 27505 init.cc:377] Logged in from keytab as oryx/127.19.202.125@KRBTEST.COM (short username oryx)
I20260810 22:39:30.307118 27505 webserver.cc:460] Webserver started at http://127.19.202.125:44253/ using document root <none> and password file <none>
I20260810 22:39:30.307283 27505 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.307333 27505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:30.308362 27505 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:30.309332 27635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.309572 27505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 22:39:30.309664 27505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "5b658c21d7a720451f2523698c3f04ee"
server_key_iv: "8c89ba4d631c7175b040674ff29f0dab"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.309739 27505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:30.326858 27505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:30.328640 27505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:30.333659 27505 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.125:42995
I20260810 22:39:30.334198 27505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
I20260810 22:39:30.334581 27690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.125:42995 every 8 connection(s)
I20260810 22:39:30.335687 27693 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:39:30.336483 27693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Bootstrap starting.
I20260810 22:39:30.344460 27693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:39:30.344816 27693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: Bootstrap complete.
I20260810 22:39:30.345309 27693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } }
I20260810 22:39:30.345441 27693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1dc88aa2aeb94b9983677db2536a7b8d, State: Initialized, Role: LEARNER
I20260810 22:39:30.345569 27693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } }
I20260810 22:39:30.345842 27696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } }
I20260810 22:39:30.346012 27696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: LEARNER
I20260810 22:39:30.346011 27693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:30.349153 27709 catalog_manager.cc:1259] Loaded cluster ID: 3ccf87bfb5c24f359435f4cd4289174a
I20260810 22:39:30.349233 27709 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: loading cluster ID for follower catalog manager: success
I20260810 22:39:30.350634 27709 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: acquiring CA information for follower catalog manager: success
I20260810 22:39:30.351639 27709 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 22:39:30.429287 27623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:30.429513 27623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:30.429548 27623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:30.433390 27623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:30.433447 27623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:30.433467 27623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:30.433485 27623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:30.433499 27623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:30.438694 27623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.124.keytab
--principal=oryx/127.19.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:36949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:30.439592 27623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:30.440694 27623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:30.449375 27719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.449429 27717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.449399 27716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:30.449895 27623 server_base.cc:1034] running on GCE node
I20260810 22:39:30.450448 27623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:30.451428 27623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:30.452605 27623 hybrid_clock.cc:648] HybridClock initialized: now 1786401570452596 us; error 28 us; skew 500 ppm
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:30.455330 27623 init.cc:377] Logged in from keytab as oryx/127.19.202.124@KRBTEST.COM (short username oryx)
I20260810 22:39:30.456130 27623 webserver.cc:460] Webserver started at http://127.19.202.124:41449/ using document root <none> and password file <none>
I20260810 22:39:30.456666 27623 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.456756 27623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:30.456988 27623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:30.458892 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/instance:
uuid: "6bbfefb06f6142a9890814aaf9536200"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "06919d5d9e2a12dc2596003c55178972"
server_key_iv: "cfb4d262a8ce5c94f3deaaf6d27a9aec"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.459451 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal/instance:
uuid: "6bbfefb06f6142a9890814aaf9536200"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "06919d5d9e2a12dc2596003c55178972"
server_key_iv: "cfb4d262a8ce5c94f3deaaf6d27a9aec"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.463064 27623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 22:39:30.465471 27727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.466516 27623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:30.466657 27623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
uuid: "6bbfefb06f6142a9890814aaf9536200"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "06919d5d9e2a12dc2596003c55178972"
server_key_iv: "cfb4d262a8ce5c94f3deaaf6d27a9aec"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.466759 27623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:30.487181 27623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:30.489987 27623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:30.490168 27623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:30.497063 27623 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.124:36949
I20260810 22:39:30.497354 27783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.124:36949 every 8 connection(s)
I20260810 22:39:30.498075 27623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
I20260810 22:39:30.500685 27784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:30.504161 27784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:39:30.505585 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 27623
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.124@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.202.124@KRBTEST.COM
I20260810 22:39:30.517920 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.506066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.19.202.126:34879)
0810 22:39:30.506201 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.506219 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.507068 (+ 849us) server_negotiation.cc:407] Connection header received
0810 22:39:30.507890 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.507892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.507939 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.508032 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.509030 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.509568 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.510650 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.510957 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.511092 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.513915 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.513919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.513930 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.513958 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.515783 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.516373 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.516376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.516378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.516470 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.516846 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.516850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.516852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.517040 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.517149 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.517628 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.517756 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:39:30.518661 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.506410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:38988)
0810 22:39:30.506942 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.506982 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.507623 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.508097 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.508102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.508542 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.508869 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.508882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.509697 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.509700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.510528 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.510537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.511129 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.511134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.511369 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.512085 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.512101 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.513750 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.515895 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.515904 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.515913 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.516267 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.516582 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.516586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.516588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.516722 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.517149 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.517156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.517388 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.517568 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.517873 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":59,"thread_start_us":154,"threads_started":1}
I20260810 22:39:30.518663 27786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.506411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54756 (local address 127.19.202.124:36949)
0810 22:39:30.506776 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.506794 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.506833 (+ 39us) server_negotiation.cc:407] Connection header received
0810 22:39:30.507623 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.507646 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.507868 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.508040 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.508834 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.509833 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.510682 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.511139 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.511338 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.514453 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.514460 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.514471 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.514497 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.516435 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.517008 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.517012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.517014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.517083 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.517369 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.517373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.517375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.517601 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.517824 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.518005 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.518133 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":242,"thread_start_us":115,"threads_started":1}
I20260810 22:39:30.519939 27784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.124@KRBTEST.COM for oryx/127.19.202.125@KRBTEST.COM
I20260810 22:39:30.527765 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.520363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59370)
0810 22:39:30.520766 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.520811 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.520880 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.521087 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.521089 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.521198 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.521277 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.521281 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.522057 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.522059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.522569 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.522577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.522996 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.522998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.523072 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.523429 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.523439 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.524613 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.526388 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.526391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.526392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.526593 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.526874 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.526876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.526878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.526975 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.527331 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.527334 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.527415 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.527521 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.527636 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346}
I20260810 22:39:30.527829 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.520575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.19.202.125:42995)
0810 22:39:30.520870 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.520884 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.520897 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:30.520955 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.520957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.520993 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.521060 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.521370 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.521970 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.522698 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.522899 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.522982 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.524758 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.524761 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.524762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.524825 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.526272 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.526702 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.526706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.526708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.526785 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.527076 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.527079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.527081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.527241 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.527342 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.527581 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.527685 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260810 22:39:30.528550 27784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.124@KRBTEST.COM for oryx/127.19.202.124@KRBTEST.COM
I20260810 22:39:30.534847 27214 heartbeater.cc:503] Master 127.19.202.126:34879 was elected leader, sending a full tablet report...
I20260810 22:39:30.535373 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.528932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.0.0.1:54768)
0810 22:39:30.529045 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.529059 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.529131 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.529310 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.529313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.529459 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.529511 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.529515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.530137 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.530140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.530579 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.530586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.530982 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.530984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.531049 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.531533 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.531543 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.532580 (+ 1037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.534032 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.534038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.534040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.534220 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.534490 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.534492 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.534494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.534570 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.534927 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.534930 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.535008 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.535141 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.535250 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 22:39:30.535558 27786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.528951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54768 (local address 127.19.202.124:36949)
0810 22:39:30.529045 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.529050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.529097 (+ 47us) server_negotiation.cc:407] Connection header received
0810 22:39:30.529191 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.529195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.529230 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.529301 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.529608 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.530033 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.530680 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.530880 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.530983 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.532704 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.532707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.532709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.532738 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.533914 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.534315 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.534319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.534321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.534392 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.534663 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.534666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.534668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.534834 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.534944 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.535209 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.535333 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 22:39:30.538561 27784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Bootstrap starting.
I20260810 22:39:30.540596 27784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:30.541335 27784 log.cc:824] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:30.542982 27784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: No bootstrap required, opened a new log
I20260810 22:39:30.545843 27784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } }
I20260810 22:39:30.546072 27784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:30.546123 27784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbfefb06f6142a9890814aaf9536200, State: Initialized, Role: FOLLOWER
I20260810 22:39:30.546519 27784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } }
I20260810 22:39:30.546938 27790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } } }
I20260810 22:39:30.547066 27790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:39:30.547603 27784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 22:39:30.552003 27802 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:39:30.552089 27802 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:39:30.555320 27623 master_runner.cc:428] Detected that this master 6bbfefb06f6142a9890814aaf9536200 is joining an existing cluster
I20260810 22:39:30.555363 27623 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.202.124:36949
I20260810 22:39:30.565879 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.557900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.19.202.126:34879)
0810 22:39:30.557999 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.558002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.558013 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:39:30.558045 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.558047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.558079 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.558174 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.558838 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.559417 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.560508 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.560893 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.560966 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.562515 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.562519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.562521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.562546 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.564145 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.564572 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.564576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.564578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.564665 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.564971 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.564975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.564977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.565224 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.565313 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.565645 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.565750 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 22:39:30.565896 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.557294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:38992)
0810 22:39:30.557605 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.557621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.557715 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.558293 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.558298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.558541 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.558701 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.558709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.559561 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.559563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.560384 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.560392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.561187 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.561190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.561331 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.561914 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.561925 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.562365 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.564258 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.564261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.564263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.564464 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.564764 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.564767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.564769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.564877 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.565376 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.565379 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.565462 (+ 83us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.565587 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.565718 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":133,"threads_started":1}
I20260810 22:39:30.571866 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 27073
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401569, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.126@KRBTEST.COM for oryx/127.19.202.124@KRBTEST.COM
I20260810 22:39:30.585067 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:44641
--local_ip_for_outbound_sockets=127.19.202.65
--tserver_master_addrs=127.19.202.126:34879
--webserver_port=38577
--webserver_interface=127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab
--principal=oryx/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:39:30.589459 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.558002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.0.0.1:54770)
0810 22:39:30.564977 (+ 6975us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.564991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.565096 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.565361 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.565364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.565563 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.565664 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.565671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.567183 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.567186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.568050 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.568060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.568813 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.568820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.568908 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.569381 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.569395 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.569755 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.588044 (+ 18289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.588047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.588049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.588261 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.588500 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.588502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.588504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.588568 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.588991 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.588993 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.589070 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.589153 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.589277 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6905,"thread_start_us":85,"threads_started":1}
I20260810 22:39:30.589740 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.557768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.19.202.125:42995)
0810 22:39:30.557890 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.557894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.558396 (+ 502us) server_negotiation.cc:407] Connection header received
0810 22:39:30.558474 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.558477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.558526 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.558608 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.559071 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.559862 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.561000 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.561410 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.561533 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.585848 (+ 24315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.585852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.585855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.585884 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.587407 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.588026 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.588031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.588035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.588138 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.588437 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.588440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.588441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.588681 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.588775 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.589386 (+ 611us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.589527 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 22:39:30.589993 27813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.557377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59372)
0810 22:39:30.558290 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.558302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.558397 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.558647 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.558651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.558848 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.558949 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.558956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.559982 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.559985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.560781 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.560813 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.561545 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.561550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.561650 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.585257 (+ 23607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.585272 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.585656 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.587536 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.587538 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.587540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.587888 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.588239 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.588245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.588247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.588338 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.588836 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.588841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.588940 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.589058 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.589212 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":100,"threads_started":1}
I20260810 22:39:30.591148 27786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.557802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54770 (local address 127.19.202.124:36949)
0810 22:39:30.557908 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.557912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.565084 (+ 7172us) server_negotiation.cc:407] Connection header received
0810 22:39:30.565165 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.565168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.565206 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.565288 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.565777 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.566498 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.568160 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.568442 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.568608 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.569893 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.569897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.569900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.569922 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.587942 (+ 18020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.588345 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.588348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.588349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.588422 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.588637 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.588640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.588641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.588872 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.588962 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.590922 (+ 1960us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.591031 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:30.593096 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.569919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.0.0.1:54772)
0810 22:39:30.570080 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.570098 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.570189 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.570573 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.570576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.570745 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.570843 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.570849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.571864 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.571867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.572914 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.572929 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.577708 (+ 4779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.577712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.577821 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.578256 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.578268 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.579524 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.591181 (+ 11657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.591186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.591190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.591481 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.591840 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.591845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.591849 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.591971 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.592518 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.592523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.592632 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.592759 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.592943 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 22:39:30.593230 27814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.569995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.19.202.124:36949)
0810 22:39:30.570304 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.570341 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.570352 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:39:30.570403 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.570404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.570440 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.570538 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.570973 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.571708 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.577282 (+ 5574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.577598 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.577705 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.579674 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.579678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.579680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.579704 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.591049 (+ 11345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.591622 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.591627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.591632 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.591726 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.592123 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.592128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.592133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.592393 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.592507 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.592872 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.593007 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":99,"threads_started":1}
I20260810 22:39:30.594100 27005 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.202.124:36949: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6bbfefb06f6142a9890814aaf9536200"
member_type: NON_VOTER
last_known_addr {
host: "127.19.202.124"
port: 36949
}
attrs {
promote: true
}
}
dest_uuid: "fcd41744f48a4235ae0ca99f5a08bd61"
cas_config_opid_index: 5
I20260810 22:39:30.594713 27005 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } }
I20260810 22:39:30.595762 27611 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.596387 27753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:30.597374 27611 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.202.124:36949
I20260810 22:39:30.597585 27611 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.597683 27611 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:30.598104 27623 master.cc:537] Master@127.19.202.124:36949 shutting down...
I20260810 22:39:30.598671 27753 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Refusing update from remote peer fcd41744f48a4235ae0ca99f5a08bd61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 22:39:30.599342 27610 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 22:39:30.601439 26992 consensus_peers.cc:487] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 -> Peer 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124:36949): Couldn't send request to peer 6bbfefb06f6142a9890814aaf9536200. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 22:39:30.604941 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.596017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59380)
0810 22:39:30.596336 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.596350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.596428 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.600935 (+ 4507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.600938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.600956 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:30.601050 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.601060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.602355 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.602358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.603549 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.603559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.604480 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.604485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.604584 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.604631 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.604707 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.604826 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260810 22:39:30.605067 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.596226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.19.202.125:42995)
0810 22:39:30.600524 (+ 4298us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.600530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.600546 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:39:30.600593 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.600598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.600670 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.600771 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:30.601179 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.601930 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.603679 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.604364 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.604483 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.604628 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.604737 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.604939 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4201}
I20260810 22:39:30.605652 27663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 LEARNER]: Refusing update from remote peer fcd41744f48a4235ae0ca99f5a08bd61: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 22:39:30.605811 27623 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:30.606081 27623 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:30.606174 27623 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: stopping tablet replica
I20260810 22:39:30.606165 27610 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 22:39:30.607901 27663 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.610476 27696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcd41744f48a4235ae0ca99f5a08bd61. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.610595 27696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: LEARNER
I20260810 22:39:30.610683 27696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: NON_VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: true } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.610752 27696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: LEARNER
I20260810 22:39:30.612622 27623 master.cc:552] Master@127.19.202.124:36949 shutdown complete.
I20260810 22:39:30.612668 27623 master_runner.cc:305] Clearing existing system tablet
I20260810 22:39:30.612931 27623 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.613029 27623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:30.614025 27610 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: attempting to promote NON_VOTER 1dc88aa2aeb94b9983677db2536a7b8d to VOTER
I20260810 22:39:30.614451 27610 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } }
I20260810 22:39:30.615573 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.615325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.0.0.1:54786)
0810 22:39:30.615444 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.615508 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:36949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 22:39:30.615690 27663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 LEARNER]: Refusing update from remote peer fcd41744f48a4235ae0ca99f5a08bd61: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 22:39:30.616425 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.616161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.0.0.1:54790)
0810 22:39:30.616306 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.616359 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:36949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 22:39:30.616417 27608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260810 22:39:30.616710 26992 consensus_peers.cc:487] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 -> Peer 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124:36949): Couldn't send request to peer 6bbfefb06f6142a9890814aaf9536200. Status: Network error: Client connection negotiation failed: client connection to 127.19.202.124:36949: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 22:39:30.618081 27609 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.618522 27663 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.619452 27609 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.619547 27609 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:30.619655 27609 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.619714 27609 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:30.620187 27817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.620357 27817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:39:30.621129 27623 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260810 22:39:30.622113 27819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.622380 27623 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:30.622642 27623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:30.622725 27623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
uuid: "6bbfefb06f6142a9890814aaf9536200"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "06919d5d9e2a12dc2596003c55178972"
server_key_iv: "cfb4d262a8ce5c94f3deaaf6d27a9aec"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.624986 27623 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:30.627274 27623 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 22:39:30.627437 27623 log.cc:1196] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 22:39:30.627744 27623 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Deleting consensus metadata
I20260810 22:39:30.627951 27623 master_runner.cc:315] Copying system tablet from 127.19.202.126:34879
I20260810 22:39:30.629118 27623 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: tablet copy: Beginning tablet copy session from remote peer at address 127.19.202.126:34879
I20260810 22:39:30.641829 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.633198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.19.202.126:34879)
0810 22:39:30.633309 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.633314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.633583 (+ 269us) server_negotiation.cc:407] Connection header received
0810 22:39:30.633694 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.633698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.633749 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.633839 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.634413 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.635215 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.636371 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.636722 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.636870 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.638212 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.638217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.638220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.638247 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.640031 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.640488 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.640492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.640494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.640585 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.640884 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.640888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.640890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.641098 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.641205 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.641545 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.641655 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:39:30.641887 27825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.633114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:38998)
0810 22:39:30.633449 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.633463 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.633605 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.633903 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.633906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.634144 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.634286 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.634294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.635341 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.635344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.636234 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.636244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.636874 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.636877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.636987 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.637607 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.637622 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.638051 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.640150 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.640154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.640156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.640383 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.640685 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.640688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.640689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.640801 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.641205 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.641208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.641353 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.641475 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.641635 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":100,"threads_started":1}
I20260810 22:39:30.642561 27028 tablet_copy_service.cc:140] P fcd41744f48a4235ae0ca99f5a08bd61: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6bbfefb06f6142a9890814aaf9536200 ({username='oryx', principal='oryx/127.19.202.124@KRBTEST.COM'} at 127.0.0.1:38998)
I20260810 22:39:30.642654 27028 tablet_copy_service.cc:161] P fcd41744f48a4235ae0ca99f5a08bd61: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6bbfefb06f6142a9890814aaf9536200 at {username='oryx', principal='oryx/127.19.202.124@KRBTEST.COM'} at 127.0.0.1:38998: session id = 6bbfefb06f6142a9890814aaf9536200-00000000000000000000000000000000
I20260810 22:39:30.643708 27028 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 22:39:30.645017 27623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 22:39:30.647279 27623 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: tablet copy: Starting download of 0 data blocks...
I20260810 22:39:30.647590 27623 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: tablet copy: Starting download of 1 WAL segments...
I20260810 22:39:30.649137 27623 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 22:39:30.650969 27028 tablet_copy_service.cc:342] P fcd41744f48a4235ae0ca99f5a08bd61: Request end of tablet copy session 6bbfefb06f6142a9890814aaf9536200-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.202.124@KRBTEST.COM'} at 127.0.0.1:38998
I20260810 22:39:30.651047 27028 tablet_copy_service.cc:433] P fcd41744f48a4235ae0ca99f5a08bd61: ending tablet copy session 6bbfefb06f6142a9890814aaf9536200-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6bbfefb06f6142a9890814aaf9536200
I20260810 22:39:30.658339 27623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:30.660058 27826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.660084 27827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.660228 27829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:30.660532 27623 server_base.cc:1034] running on GCE node
I20260810 22:39:30.660670 27623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:30.660961 27623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:30.662173 27623 hybrid_clock.cc:648] HybridClock initialized: now 1786401570662160 us; error 30 us; skew 500 ppm
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:30.663847 27623 init.cc:377] Logged in from keytab as oryx/127.19.202.124@KRBTEST.COM (short username oryx)
I20260810 22:39:30.664332 27623 webserver.cc:460] Webserver started at http://127.19.202.124:37185/ using document root <none> and password file <none>
I20260810 22:39:30.664516 27623 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.664568 27623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:30.665936 27623 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:30.666919 27836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.667203 27623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:30.667277 27623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
uuid: "6bbfefb06f6142a9890814aaf9536200"
format_stamp: "Formatted at 2026-08-10 22:39:30 on dist-test-slave-1x4g"
server_key: "06919d5d9e2a12dc2596003c55178972"
server_key_iv: "cfb4d262a8ce5c94f3deaaf6d27a9aec"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.667353 27623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:30.684288 27623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:30.686636 27623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:30.692124 27623 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.124:36949
I20260810 22:39:30.692507 27891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.124:36949 every 8 connection(s)
I20260810 22:39:30.692842 27623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
I20260810 22:39:30.694759 27892 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:39:30.695820 27892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Bootstrap starting.
I20260810 22:39:30.707304 27892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:39:30.707679 27892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Bootstrap complete.
I20260810 22:39:30.708280 27892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } }
I20260810 22:39:30.708441 27892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6bbfefb06f6142a9890814aaf9536200, State: Initialized, Role: LEARNER
I20260810 22:39:30.708562 27892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } }
I20260810 22:39:30.709055 27892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:30.712563 27908 catalog_manager.cc:1259] Loaded cluster ID: 3ccf87bfb5c24f359435f4cd4289174a
I20260810 22:39:30.712674 27908 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: loading cluster ID for follower catalog manager: success
I20260810 22:39:30.713111 27895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:30.713261 27895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: This master's current role is: LEARNER
I20260810 22:39:30.715592 27908 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: acquiring CA information for follower catalog manager: success
I20260810 22:39:30.717155 27908 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 22:39:30.734680 27815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:30.735101 27815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:30.735212 27815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:30.742029 27815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:30.742096 27815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:30.742193 27815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:30.746327 27355 heartbeater.cc:503] Master 127.19.202.126:34879 was elected leader, sending a full tablet report...
I20260810 22:39:30.752749 27815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab
--principal=oryx/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:44641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=38577
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:30.754189 27815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:30.755453 27815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:30.768194 27914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.768260 27913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.768323 27916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:30.768693 27815 server_base.cc:1034] running on GCE node
I20260810 22:39:30.769222 27815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:30.769943 27815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:30.771134 27815 hybrid_clock.cc:648] HybridClock initialized: now 1786401570771110 us; error 34 us; skew 500 ppm
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:30.775571 27815 init.cc:377] Logged in from keytab as oryx/127.19.202.65@KRBTEST.COM (short username oryx)
I20260810 22:39:30.776640 27815 webserver.cc:460] Webserver started at http://127.19.202.65:38577/ using document root <none> and password file <none>
I20260810 22:39:30.777283 27815 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.777390 27815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:30.781318 27815 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:30.783538 27924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.785279 27815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:30.785390 27815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "8d0cb262e1124afa902aa295415cc336"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "934cf9bf4bda22890d1b0cd603a265d0"
server_key_iv: "e7b42e304d330f04c55dbe9ba78e7934"
server_key_version: "encryptionkey@0"
I20260810 22:39:30.786139 27815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:30.817647 27815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:30.820642 27815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:30.820856 27815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:30.821543 27815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:30.822428 27815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:30.822472 27815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.822510 27815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:30.822538 27815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:30.840597 27815 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:44641
I20260810 22:39:30.840620 28043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:44641 every 8 connection(s)
I20260810 22:39:30.841992 27815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:30.844663 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 27815
I20260810 22:39:30.844872 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 27219
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.124@KRBTEST.COM
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.125@KRBTEST.COM
I20260810 22:39:30.858429 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:40455
--local_ip_for_outbound_sockets=127.19.202.66
--tserver_master_addrs=127.19.202.126:34879
--webserver_port=41067
--webserver_interface=127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab
--principal=oryx/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:39:30.860358 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.843935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:53735 (local address 127.19.202.126:34879)
0810 22:39:30.844097 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.844104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.846381 (+ 2277us) server_negotiation.cc:407] Connection header received
0810 22:39:30.846518 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.846524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.846583 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.846666 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.848381 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.849091 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.851137 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.851540 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.851631 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.855521 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.855527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.855534 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.855567 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.857790 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.858565 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.858568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.858570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.858655 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.859068 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.859073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.859074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.859277 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.859385 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.859977 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.860191 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:30.861402 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.844404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:53735)
0810 22:39:30.846277 (+ 1873us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.846293 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.846408 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.846976 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.846987 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.847593 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.848225 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.848244 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.849242 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.849247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.850995 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.851010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.851843 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.851849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.852066 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.853211 (+ 1145us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.853234 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.855313 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.857920 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.857929 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.857943 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.858447 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.858766 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.858770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.858774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.858963 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.859385 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.859392 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.859695 (+ 303us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.859926 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.860285 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1795,"spinlock_wait_cycles":38656,"thread_start_us":188,"threads_started":1}
I20260810 22:39:30.862675 28045 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34879
I20260810 22:39:30.862910 28045 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:30.863247 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.851877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:42737 (local address 127.19.202.125:42995)
0810 22:39:30.852077 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.852090 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.852112 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:39:30.852172 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.852182 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.852233 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.852308 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.853049 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.853768 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.854902 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.855253 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.855339 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.858823 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.858829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.858834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.858875 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.860525 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.861328 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.861345 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.861350 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.861456 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.862078 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.862082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.862084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.862345 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.862453 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.862942 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.863069 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 22:39:30.863353 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.844336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.19.202.65:41691)
0810 22:39:30.844824 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.844878 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.845966 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.851655 (+ 5689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.851658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.851853 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.852209 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.852217 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.853589 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.853592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.854430 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.854438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.855050 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.855053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.855154 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.856009 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.856027 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.858470 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.861404 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.861407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.861410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.861651 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.861979 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.861985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.861988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.862076 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.862592 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.862595 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.862701 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.862805 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.862921 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"spinlock_wait_cycles":5376,"thread_start_us":136,"threads_started":1}
I20260810 22:39:30.863483 28045 heartbeater.cc:511] Master 127.19.202.126:34879 requested a full tablet report, sending...
I20260810 22:39:30.864118 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.845321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:41691 (local address 127.19.202.124:36949)
0810 22:39:30.851446 (+ 6125us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:30.851449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:30.851462 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:30.851498 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:30.851500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:30.851537 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:30.851641 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:30.852353 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.853454 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.854581 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.854935 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.855374 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.858616 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:30.858622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:30.858625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:30.858647 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:30.861239 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.861777 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.861779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.861781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.861874 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:30.862190 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:30.862193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:30.862194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:30.862469 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:30.863731 (+ 1262us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:30.863804 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 22:39:30.863945 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6058,"thread_start_us":647,"threads_started":1}
I20260810 22:39:30.864475 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:30.845353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.19.202.65:42737)
0810 22:39:30.848674 (+ 3321us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:30.848692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:30.848855 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:30.852492 (+ 3637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:30.852495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:30.852749 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:30.852892 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.852898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.853906 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.853909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.854712 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:30.854719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:30.855499 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:30.855502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:30.855569 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:30.856195 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:30.856215 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:30.858444 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:30.860922 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.860928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.860933 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.861200 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.861832 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:30.861838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:30.861841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:30.861955 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:30.862543 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:30.862548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:30.862659 (+ 111us) client_negotiation.cc:769] Sending connection context
0810 22:39:30.863803 (+ 1144us) client_negotiation.cc:240] Negotiation successful
0810 22:39:30.864008 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3236,"thread_start_us":622,"threads_started":1}
I20260810 22:39:30.865000 28044 heartbeater.cc:347] Connected to a master server at 127.19.202.124:36949
I20260810 22:39:30.865044 27005 ts_manager.cc:194] Re-registered known tserver with Master: 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641)
I20260810 22:39:30.865116 28044 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:30.865312 28044 heartbeater.cc:511] Master 127.19.202.124:36949 requested a full tablet report, sending...
I20260810 22:39:30.865690 27005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:53735
I20260810 22:39:30.865994 28046 heartbeater.cc:347] Connected to a master server at 127.19.202.125:42995
I20260810 22:39:30.866077 28046 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:30.866271 28046 heartbeater.cc:511] Master 127.19.202.125:42995 requested a full tablet report, sending...
I20260810 22:39:30.866442 27847 ts_manager.cc:194] Registered new tserver with Master: 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641)
I20260810 22:39:30.867623 27653 ts_manager.cc:194] Registered new tserver with Master: 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641)
I20260810 22:39:30.962706 27500 heartbeater.cc:503] Master 127.19.202.126:34879 was elected leader, sending a full tablet report...
W20260810 22:39:30.972445 28052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:30.972658 28052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:30.972725 28052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:30.976483 28052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:30.976529 28052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:30.976593 28052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:30.981241 28052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab
--principal=oryx/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:40455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=41067
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:30.982097 28052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:30.982990 28052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:30.992242 28060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.992283 28058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:30.992274 28057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:30.992461 28052 server_base.cc:1034] running on GCE node
I20260810 22:39:30.992856 28052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:30.993381 28052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:30.994535 28052 hybrid_clock.cc:648] HybridClock initialized: now 1786401570994507 us; error 39 us; skew 500 ppm
Aug 10 22:39:30 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:30.997201 28052 init.cc:377] Logged in from keytab as oryx/127.19.202.66@KRBTEST.COM (short username oryx)
I20260810 22:39:30.997946 28052 webserver.cc:460] Webserver started at http://127.19.202.66:41067/ using document root <none> and password file <none>
I20260810 22:39:30.998484 28052 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:30.998607 28052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:31.002126 28052 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:31.003989 28067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:31.005232 28052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:31.005360 28052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "be93acea2f3f4a8caad402f6374d483c"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "7ab76306532591dc18c057ea90bf9f60"
server_key_iv: "1d5678a5cf7efd3d40689c1a8269d9ca"
server_key_version: "encryptionkey@0"
I20260810 22:39:31.005856 28052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:31.017297 28052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:31.020008 28052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:31.020182 28052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:31.020830 28052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:31.021697 28052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:31.021750 28052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:31.021814 28052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:31.021845 28052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:31.030723 28052 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:40455
I20260810 22:39:31.030795 28188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:40455 every 8 connection(s)
I20260810 22:39:31.031651 28052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:31.037830 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 28052
I20260810 22:39:31.037988 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 27360
Aug 10 22:39:31 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
Aug 10 22:39:31 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.124@KRBTEST.COM
I20260810 22:39:31.050130 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.033780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:35777 (local address 127.19.202.126:34879)
0810 22:39:31.034094 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.034100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.035095 (+ 995us) server_negotiation.cc:407] Connection header received
0810 22:39:31.035763 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.035766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.035821 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.035914 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.037484 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.038287 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.041866 (+ 3579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.042189 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.042264 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.045474 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.045478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.045482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.045505 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.047595 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.048316 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.048320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.048324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.048431 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.048897 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.048903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.048905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.049145 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.049227 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.049842 (+ 615us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.049981 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260810 22:39:31.050865 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.034921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:57361 (local address 127.19.202.124:36949)
0810 22:39:31.035059 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.035064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.036021 (+ 957us) server_negotiation.cc:407] Connection header received
0810 22:39:31.036200 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.036203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.036303 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.036416 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.037484 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.038287 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.044151 (+ 5864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.044556 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.044641 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.047221 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.047225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.047228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.047259 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.049079 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.049607 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.049610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.049612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.049691 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.049980 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.049983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.049984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.050175 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.050256 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.050609 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.050729 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 22:39:31.051878 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.034793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.19.202.66:57361)
0810 22:39:31.035942 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.035955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.036093 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.036435 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.036438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.036726 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:31.037314 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.037330 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.043062 (+ 5732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.043067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.043979 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.043993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.044836 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.044840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.044935 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.045466 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:31.045478 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:31.047048 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:31.049210 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.049213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.049215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.049494 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.049795 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.049797 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.049800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.049875 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.050325 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:31.050330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:31.050433 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.051053 (+ 620us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.051176 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1088,"mutex_wait_us":60,"thread_start_us":115,"threads_started":1}
I20260810 22:39:31.051885 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.033751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:35777)
0810 22:39:31.034691 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.034726 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.035569 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.036075 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.036085 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.036597 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:31.037317 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.037330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.038527 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.038532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.041736 (+ 3204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.041749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.042466 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.042472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.042682 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.043280 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:31.043298 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:31.045288 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:31.047806 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.047812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.047827 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.048189 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.048560 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.048563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.048567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.048744 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.049292 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:31.049298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:31.049584 (+ 286us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.050509 (+ 925us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.050786 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":92,"spinlock_wait_cycles":13184,"thread_start_us":145,"threads_started":1}
I20260810 22:39:31.053087 28189 heartbeater.cc:347] Connected to a master server at 127.19.202.124:36949
I20260810 22:39:31.053098 28190 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34879
I20260810 22:39:31.053339 28190 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:31.053344 28189 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:31.053915 28189 heartbeater.cc:511] Master 127.19.202.124:36949 requested a full tablet report, sending...
I20260810 22:39:31.053920 28190 heartbeater.cc:511] Master 127.19.202.126:34879 requested a full tablet report, sending...
I20260810 22:39:31.055109 27005 ts_manager.cc:194] Re-registered known tserver with Master: be93acea2f3f4a8caad402f6374d483c (127.19.202.66:40455)
I20260810 22:39:31.055666 27005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:35777
Aug 10 22:39:31 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401570, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.125@KRBTEST.COM
I20260810 22:39:31.057359 27847 ts_manager.cc:194] Registered new tserver with Master: be93acea2f3f4a8caad402f6374d483c (127.19.202.66:40455)
I20260810 22:39:31.059643 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.033751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.19.202.66:32945)
0810 22:39:31.035055 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.035068 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.035582 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.036076 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.036083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.036720 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:31.037312 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.037328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.049498 (+ 12170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.049504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.050638 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.050653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.051601 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.051609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.051738 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.052275 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:31.052296 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:31.056091 (+ 3795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:31.058230 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.058232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.058233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.058436 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.058742 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.058744 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.058745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.058814 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.059284 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:31.059286 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:31.059350 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.059424 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.059523 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1220,"mutex_wait_us":63,"spinlock_wait_cycles":33920,"thread_start_us":142,"threads_started":1}
I20260810 22:39:31.059881 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.033505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:32945 (local address 127.19.202.125:42995)
0810 22:39:31.033689 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.033695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.035159 (+ 1464us) server_negotiation.cc:407] Connection header received
0810 22:39:31.035751 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.035754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.035808 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.035907 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.037462 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.038227 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.050812 (+ 12585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.051275 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.051376 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.056245 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.056252 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.056258 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.056292 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.058122 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.058541 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.058547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.058551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.058651 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.058915 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.058921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.058926 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.059189 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.059295 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.059514 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.059646 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:39:31.060413 28191 heartbeater.cc:347] Connected to a master server at 127.19.202.125:42995
I20260810 22:39:31.060503 28191 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:31.060698 28191 heartbeater.cc:511] Master 127.19.202.125:42995 requested a full tablet report, sending...
I20260810 22:39:31.061520 27653 ts_manager.cc:194] Registered new tserver with Master: be93acea2f3f4a8caad402f6374d483c (127.19.202.66:40455)
I20260810 22:39:31.061955 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:39199
--local_ip_for_outbound_sockets=127.19.202.67
--tserver_master_addrs=127.19.202.126:34879
--webserver_port=42277
--webserver_interface=127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab
--principal=oryx/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:31.172075 28196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:31.172307 28196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:31.172379 28196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:31.176308 28196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:31.176357 28196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:31.176450 28196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:31.180891 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.177564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.19.202.124:36949)
0810 22:39:31.177703 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.177708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.177901 (+ 193us) server_negotiation.cc:407] Connection header received
0810 22:39:31.178096 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.178100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.178153 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.178247 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:31.178504 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.178978 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.179938 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.180344 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.180481 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.180625 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.180658 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.180720 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:39:31.180893 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.177483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.0.0.1:54806)
0810 22:39:31.177811 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.177823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.177967 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.178264 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.178268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.178286 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:31.178396 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.178403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.179094 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.179098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.179820 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.179827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.180472 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.180476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.180559 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.180581 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.180644 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.180720 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":89,"threads_started":1}
I20260810 22:39:31.181690 28196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab
--principal=oryx/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:39199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=42277
--tserver_master_addrs=127.19.202.126:34879,127.19.202.125:42995,127.19.202.124:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:31.181706 27856 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 22:39:31.182091 27895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcd41744f48a4235ae0ca99f5a08bd61. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:31.182230 27895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: This master's current role is: LEARNER
I20260810 22:39:31.182845 28196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:31.183832 28196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:31.193203 28204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:31.193193 28206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:31.193193 28203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:31.193464 28196 server_base.cc:1034] running on GCE node
I20260810 22:39:31.193919 28196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:31.194511 28196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:31.195667 28196 hybrid_clock.cc:648] HybridClock initialized: now 1786401571195631 us; error 46 us; skew 500 ppm
Aug 10 22:39:31 dist-test-slave-1x4g krb5kdc[26958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401571, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:31.198390 28196 init.cc:377] Logged in from keytab as oryx/127.19.202.67@KRBTEST.COM (short username oryx)
I20260810 22:39:31.199273 28196 webserver.cc:460] Webserver started at http://127.19.202.67:42277/ using document root <none> and password file <none>
I20260810 22:39:31.199766 28196 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:31.199837 28196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:31.203466 28196 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260810 22:39:31.205435 28214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:31.206614 28196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:31.206729 28196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "6361f8d43c044c939eaa4585da651f4d"
format_stamp: "Formatted at 2026-08-10 22:39:29 on dist-test-slave-1x4g"
server_key: "768b8b75da6ebb23f013a04f84436959"
server_key_iv: "a94817ec1daeb61f8cec379398407c80"
server_key_version: "encryptionkey@0"
I20260810 22:39:31.207216 28196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:31.217160 28196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:31.219745 28196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:31.219934 28196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:31.220486 28196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:31.221343 28196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:31.221397 28196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:31.221460 28196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:31.221489 28196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:31.229815 28196 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:39199
I20260810 22:39:31.229859 28335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:39199 every 8 connection(s)
I20260810 22:39:31.230702 28196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:31.238126 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 28196
Aug 10 22:39:31 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401571, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.124@KRBTEST.COM
I20260810 22:39:31.246523 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.234144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:59787 (local address 127.19.202.124:36949)
0810 22:39:31.234264 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.234268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.234375 (+ 107us) server_negotiation.cc:407] Connection header received
0810 22:39:31.235001 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.235004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.235066 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.235155 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.236298 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.237152 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.238556 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.238854 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.238956 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.241970 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.241974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.241977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.242002 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.243870 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.244466 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.244469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.244471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.244553 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.244943 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.244946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.244948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.245179 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.245262 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.246228 (+ 966us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.246356 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 22:39:31 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401571, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.125@KRBTEST.COM
I20260810 22:39:31.251220 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.232970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.19.202.67:38637)
0810 22:39:31.233537 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.233596 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.234787 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.235320 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.235323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.235900 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:31.236283 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.236290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.237349 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.237362 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.238606 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.238617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.239200 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.239203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.239422 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.245694 (+ 6272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:31.245705 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:31.247229 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:31.249113 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.249115 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.249117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.249323 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.249593 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.249596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.249598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.249667 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.250085 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:31.250088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:31.250194 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.250285 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.250403 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":107,"spinlock_wait_cycles":5760,"thread_start_us":136,"threads_started":1}
Aug 10 22:39:31 dist-test-slave-1x4g krb5kdc[26958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401571, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:31.251839 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.232396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:38637 (local address 127.19.202.125:42995)
0810 22:39:31.232520 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.232525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.233697 (+ 1172us) server_negotiation.cc:407] Connection header received
0810 22:39:31.235050 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.235053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.235117 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.235180 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.236443 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.237082 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.238717 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.239011 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.239071 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.247340 (+ 8269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.247343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.247346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.247369 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.249004 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.249409 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.249423 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.249425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.249493 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.249746 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.249748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.249750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.249945 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.250040 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.251587 (+ 1547us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.251699 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:31.252933 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.234027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.19.202.67:59787)
0810 22:39:31.234272 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.234284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.234792 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.235313 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.235319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.235689 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:31.236149 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.236163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.237330 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.237333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.238405 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.238416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.239164 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.239171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.239344 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.239917 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:31.239940 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:31.241804 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:31.244014 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.244019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.244032 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.244345 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.244643 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.244646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.244648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.244833 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.245309 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:31.245315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:31.245609 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.245844 (+ 235us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.247004 (+ 1160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260810 22:39:31.253506 28338 heartbeater.cc:347] Connected to a master server at 127.19.202.125:42995
I20260810 22:39:31.253818 28338 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:31.254482 28338 heartbeater.cc:511] Master 127.19.202.125:42995 requested a full tablet report, sending...
I20260810 22:39:31.254762 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.240918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.19.202.126:34879)
0810 22:39:31.241106 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.241110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.242655 (+ 1545us) server_negotiation.cc:407] Connection header received
0810 22:39:31.242805 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.242808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.242884 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.243270 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.244419 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.245231 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.246287 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.246610 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.246695 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.249595 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.249600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.249606 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.249634 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.251818 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.252623 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.252627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.252629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.252732 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.253152 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.253158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.253163 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.253400 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.253505 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.253933 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.254648 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":58,"threads_started":1}
I20260810 22:39:31.255834 28336 heartbeater.cc:347] Connected to a master server at 127.19.202.124:36949
I20260810 22:39:31.255925 28336 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:31.256072 28336 heartbeater.cc:511] Master 127.19.202.124:36949 requested a full tablet report, sending...
I20260810 22:39:31.256682 27653 ts_manager.cc:194] Registered new tserver with Master: 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199)
I20260810 22:39:31.257742 27847 ts_manager.cc:194] Registered new tserver with Master: 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199)
I20260810 22:39:31.260170 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.242039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54818 (local address 127.19.202.124:36949)
0810 22:39:31.243765 (+ 1726us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.243770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.243781 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:39:31.243816 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.243820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.243885 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.244911 (+ 1026us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.245005 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.245672 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.246735 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.247216 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.247319 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.249660 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.249666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.249671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.249696 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.253238 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.257223 (+ 3985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.257226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.257228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.257361 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.258460 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.258463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.258465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.258655 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.259826 (+ 1171us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.259910 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.260039 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1653,"thread_start_us":104,"threads_started":1}
I20260810 22:39:31.261016 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.232395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:36289 (local address 127.19.202.126:34879)
0810 22:39:31.232520 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.232525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.233696 (+ 1171us) server_negotiation.cc:407] Connection header received
0810 22:39:31.235053 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.235056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.235110 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.235177 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.236395 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.237161 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.238659 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.238978 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.239053 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.253385 (+ 14332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.253389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.253391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.253409 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.255863 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.258046 (+ 2183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.258049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.258050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.258171 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.258771 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.258774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.258775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.258973 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.259047 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.260759 (+ 1712us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.260890 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:31.258170 27005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 22:39:31.261477 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.232970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:36289)
0810 22:39:31.233537 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.233596 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.234787 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.235320 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.235323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.235900 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:31.236272 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.236282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.237384 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.237388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.238526 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.238537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.239200 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.239203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.248630 (+ 9427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.250116 (+ 1486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:31.250132 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:31.252281 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:31.256011 (+ 3730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.256014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.256016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.257943 (+ 1927us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.258288 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:31.258291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:31.258292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:31.258370 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:31.259104 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:31.259106 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:31.259183 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.261140 (+ 1957us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.261272 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":71,"spinlock_wait_cycles":54016,"thread_start_us":133,"threads_started":1}
I20260810 22:39:31.262434 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.241482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.19.202.125:42995)
0810 22:39:31.241724 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.241728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.243113 (+ 1385us) server_negotiation.cc:407] Connection header received
0810 22:39:31.243230 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.253261 (+ 10031us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.253318 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.254579 (+ 1261us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.254680 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.255429 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.256497 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.256861 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.257563 (+ 702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.258678 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.258681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.258684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.258708 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.260472 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.260905 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.260909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.260910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.260992 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.261267 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.261271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.261272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.261437 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.261562 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.262172 (+ 610us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.262310 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":109,"threads_started":1}
I20260810 22:39:31.262450 28337 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34879
I20260810 22:39:31.262549 28337 heartbeater.cc:464] Registering TS with master...
W20260810 22:39:31.262547 27005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:31.262696 28337 heartbeater.cc:511] Master 127.19.202.126:34879 requested a full tablet report, sending...
I20260810 22:39:31.263346 27004 ts_manager.cc:194] Re-registered known tserver with Master: 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199)
I20260810 22:39:31.263993 27004 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:36289
I20260810 22:39:31.265478 27608 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: NON_VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: true } } }
I20260810 22:39:31.265621 27608 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:31.269062 28356 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: attempting to promote NON_VOTER 6bbfefb06f6142a9890814aaf9536200 to VOTER
I20260810 22:39:31.269394 28356 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:31.270654 27663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Refusing update from remote peer fcd41744f48a4235ae0ca99f5a08bd61: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 22:39:31.270979 28355 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 22:39:31.272264 27608 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.272554 27860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 LEARNER]: Refusing update from remote peer fcd41744f48a4235ae0ca99f5a08bd61: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 22:39:31.272748 27663 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.273711 28356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 22:39:31.273785 27608 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.273851 27608 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:31.273983 27608 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.274067 27608 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:31.275070 28367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.275216 28367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:39:31.275002 27860 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.276875 27895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.276989 27895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:39:31.278872 27608 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:31.279011 27608 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:31.282997 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.277378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39199 (local address 127.0.0.1:49392)
0810 22:39:31.277546 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.277561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.277629 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.278360 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.278364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.278382 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:31.278499 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.278507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.280278 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.280282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.281181 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.281190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.282308 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.282312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.282393 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.282542 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.282641 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.282711 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":99}
I20260810 22:39:31.283250 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.277578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.19.202.67:39199)
0810 22:39:31.277895 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.277904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.277922 (+ 18us) server_negotiation.cc:407] Connection header received
0810 22:39:31.278003 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.278009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.278195 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.279056 (+ 861us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:31.279339 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.280177 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.281428 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.282138 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.282301 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.282463 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.282684 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.282767 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":127,"threads_started":1}
I20260810 22:39:31.284909 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.278214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.19.202.65:44641)
0810 22:39:31.279942 (+ 1728us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.279981 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.280006 (+ 25us) server_negotiation.cc:407] Connection header received
0810 22:39:31.280080 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.280084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.280247 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.280445 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:31.280600 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.281747 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.283291 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.284229 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.284372 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.284513 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.284607 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.284721 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1619,"thread_start_us":208,"threads_started":1}
I20260810 22:39:31.284763 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.277993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44641 (local address 127.0.0.1:60868)
0810 22:39:31.278715 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.278728 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.278841 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.280419 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.280422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.280439 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:31.280510 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.280515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.281869 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.281872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.283128 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.283142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.284368 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.284372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.284486 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.284510 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.284578 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.284644 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260810 22:39:31.285756 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.278528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.19.202.66:40455)
0810 22:39:31.279147 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.279191 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.280148 (+ 957us) server_negotiation.cc:407] Connection header received
0810 22:39:31.280218 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.280221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.280384 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.280552 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:31.281057 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.282129 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.283624 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.284615 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.284730 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.284914 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.285461 (+ 547us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.285560 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":174,"threads_started":1}
I20260810 22:39:31.286083 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.278294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:40455 (local address 127.0.0.1:33852)
0810 22:39:31.279514 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.279527 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.279644 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.280698 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.280701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.280717 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:31.280911 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.280920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.282238 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.282244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.283484 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.283494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.285141 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.285145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.285296 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.285320 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.285426 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.285499 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"mutex_wait_us":110,"thread_start_us":140,"threads_started":1}
I20260810 22:39:31.286408 28266 tablet_service.cc:1467] Processing CreateTablet for tablet 501d921d286e4db3b591471ed5d872a4 (DEFAULT_TABLE table=test-table [id=6c7f0d10953c45c5ab48eca2db18e2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:31.287572 28266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501d921d286e4db3b591471ed5d872a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:31.288019 27974 tablet_service.cc:1467] Processing CreateTablet for tablet 501d921d286e4db3b591471ed5d872a4 (DEFAULT_TABLE table=test-table [id=6c7f0d10953c45c5ab48eca2db18e2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:31.288856 27974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501d921d286e4db3b591471ed5d872a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:31.288769 28119 tablet_service.cc:1467] Processing CreateTablet for tablet 501d921d286e4db3b591471ed5d872a4 (DEFAULT_TABLE table=test-table [id=6c7f0d10953c45c5ab48eca2db18e2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:31.289724 28119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501d921d286e4db3b591471ed5d872a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:31.296411 28373 tablet_bootstrap.cc:492] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Bootstrap starting.
I20260810 22:39:31.299083 28373 tablet_bootstrap.cc:654] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:31.299863 28375 tablet_bootstrap.cc:492] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Bootstrap starting.
I20260810 22:39:31.301090 28373 log.cc:824] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:31.301329 28374 tablet_bootstrap.cc:492] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Bootstrap starting.
I20260810 22:39:31.302922 28375 tablet_bootstrap.cc:654] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:31.303601 28373 tablet_bootstrap.cc:492] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: No bootstrap required, opened a new log
I20260810 22:39:31.303718 28374 tablet_bootstrap.cc:654] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:31.303880 28373 ts_tablet_manager.cc:1397] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 22:39:31.304109 28375 log.cc:824] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:31.305035 28374 log.cc:824] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:31.306622 28374 tablet_bootstrap.cc:492] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: No bootstrap required, opened a new log
I20260810 22:39:31.306622 28375 tablet_bootstrap.cc:492] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: No bootstrap required, opened a new log
I20260810 22:39:31.306807 28374 ts_tablet_manager.cc:1397] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 22:39:31.306862 28375 ts_tablet_manager.cc:1397] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 22:39:31.310398 28373 raft_consensus.cc:348] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.310698 28373 raft_consensus.cc:374] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:31.310770 28373 raft_consensus.cc:729] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d0cb262e1124afa902aa295415cc336, State: Initialized, Role: FOLLOWER
I20260810 22:39:31.311517 28373 consensus_queue.cc:260] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.312834 28373 ts_tablet_manager.cc:1428] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Time spent starting tablet: real 0.009s user 0.007s sys 0.004s
I20260810 22:39:31.312990 28374 raft_consensus.cc:348] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.313225 28374 raft_consensus.cc:374] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:31.313241 28045 heartbeater.cc:503] Master 127.19.202.126:34879 was elected leader, sending a full tablet report...
I20260810 22:39:31.313272 28374 raft_consensus.cc:729] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6361f8d43c044c939eaa4585da651f4d, State: Initialized, Role: FOLLOWER
I20260810 22:39:31.313706 28374 consensus_queue.cc:260] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.314343 28375 raft_consensus.cc:348] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.314597 28375 raft_consensus.cc:374] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:31.314651 28375 raft_consensus.cc:729] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93acea2f3f4a8caad402f6374d483c, State: Initialized, Role: FOLLOWER
I20260810 22:39:31.314725 28374 ts_tablet_manager.cc:1428] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260810 22:39:31.314947 28337 heartbeater.cc:503] Master 127.19.202.126:34879 was elected leader, sending a full tablet report...
I20260810 22:39:31.315112 28375 consensus_queue.cc:260] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.316267 28375 ts_tablet_manager.cc:1428] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Time spent starting tablet: real 0.009s user 0.000s sys 0.010s
I20260810 22:39:31.319365 28190 heartbeater.cc:503] Master 127.19.202.126:34879 was elected leader, sending a full tablet report...
W20260810 22:39:31.343936 28047 tablet.cc:2369] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:39:31.418421 28381 raft_consensus.cc:482] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:31.418645 28381 raft_consensus.cc:504] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.419932 28381 leader_election.cc:288] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641), 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199)
I20260810 22:39:31.423514 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.420138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39199 (local address 127.19.202.66:48741)
0810 22:39:31.420388 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.420403 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.420512 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.420748 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.420751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.420765 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:31.420890 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.420900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.421579 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.421584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.422286 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.422294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.423044 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.423047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.423145 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.423167 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.423279 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.423349 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":103}
I20260810 22:39:31.423538 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.420251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:48741 (local address 127.19.202.67:39199)
0810 22:39:31.420375 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.420378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.420474 (+ 96us) server_negotiation.cc:407] Connection header received
0810 22:39:31.420621 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.420624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.420663 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.420737 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:31.420998 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.421473 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.422424 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.422928 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.423051 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.423102 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.423309 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.423401 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 22:39:31.423629 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.420124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44641 (local address 127.19.202.66:51595)
0810 22:39:31.420235 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:31.420248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:31.420332 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:31.420596 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:31.420598 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:31.420614 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:31.420880 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.420887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.421592 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.421595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.422494 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:31.422503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.423157 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.423160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:31.423313 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.423334 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 22:39:31.423407 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 22:39:31.423470 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1}
I20260810 22:39:31.423632 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.420200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:51595 (local address 127.19.202.65:44641)
0810 22:39:31.420307 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.420311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.420324 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:31.420461 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.420463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.420503 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.420575 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:31.420987 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.421477 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.422617 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.422993 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.423149 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.423195 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.423444 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.423514 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:39:31.424199 28287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501d921d286e4db3b591471ed5d872a4" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6361f8d43c044c939eaa4585da651f4d" is_pre_election: true
I20260810 22:39:31.424343 27994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501d921d286e4db3b591471ed5d872a4" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d0cb262e1124afa902aa295415cc336" is_pre_election: true
I20260810 22:39:31.424522 28287 raft_consensus.cc:2454] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93acea2f3f4a8caad402f6374d483c in term 0.
I20260810 22:39:31.424656 27994 raft_consensus.cc:2454] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93acea2f3f4a8caad402f6374d483c in term 0.
I20260810 22:39:31.425127 28068 leader_election.cc:302] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6361f8d43c044c939eaa4585da651f4d, be93acea2f3f4a8caad402f6374d483c; no voters:
I20260810 22:39:31.425410 28381 raft_consensus.cc:2781] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:31.425499 28381 raft_consensus.cc:482] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:31.425550 28381 raft_consensus.cc:3037] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:31.426836 28381 raft_consensus.cc:504] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.427170 28381 leader_election.cc:288] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 election: Requested vote from peers 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641), 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199)
I20260810 22:39:31.427457 27994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501d921d286e4db3b591471ed5d872a4" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d0cb262e1124afa902aa295415cc336"
I20260810 22:39:31.427567 27994 raft_consensus.cc:3037] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:31.428740 27994 raft_consensus.cc:2454] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93acea2f3f4a8caad402f6374d483c in term 1.
I20260810 22:39:31.428934 28287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501d921d286e4db3b591471ed5d872a4" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6361f8d43c044c939eaa4585da651f4d"
I20260810 22:39:31.429056 28287 raft_consensus.cc:3037] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:31.429183 28072 leader_election.cc:302] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d0cb262e1124afa902aa295415cc336, be93acea2f3f4a8caad402f6374d483c; no voters:
I20260810 22:39:31.429354 28381 raft_consensus.cc:2781] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:31.429595 28381 raft_consensus.cc:686] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 1 LEADER]: Becoming Leader. State: Replica: be93acea2f3f4a8caad402f6374d483c, State: Running, Role: LEADER
I20260810 22:39:31.429913 28381 consensus_queue.cc:237] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } }
I20260810 22:39:31.430815 28287 raft_consensus.cc:2454] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93acea2f3f4a8caad402f6374d483c in term 1.
I20260810 22:39:31.433323 27005 catalog_manager.cc:5495] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c reported cstate change: term changed from 0 to 1, leader changed from <none> to be93acea2f3f4a8caad402f6374d483c (127.19.202.66). New cstate: current_term: 1 leader_uuid: "be93acea2f3f4a8caad402f6374d483c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } health_report { overall_health: HEALTHY } } }
I20260810 22:39:31.471988 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.469063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.19.202.66:40455)
0810 22:39:31.469177 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.469180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.469193 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:31.469283 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.469285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.469333 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.469419 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:31.469613 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.470060 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.470792 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.470974 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.471042 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.471269 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:31.471651 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:31.471735 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.471785 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.471828 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:39:31.477995 27994 raft_consensus.cc:1261] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 1 FOLLOWER]: Refusing update from remote peer be93acea2f3f4a8caad402f6374d483c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:31.478109 28287 raft_consensus.cc:1261] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Refusing update from remote peer be93acea2f3f4a8caad402f6374d483c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:31.478691 28381 consensus_queue.cc:1035] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:31.478924 28382 consensus_queue.cc:1035] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 22:39:31.482038 28339 tablet.cc:2369] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:39:31.486980 28385 mvcc.cc:205] Tried to move back new op lower bound from 7317100836766928896 to 7317100836579188736. Current Snapshot: MvccSnapshot[applied={T|T < 7317100836766928896}]
I20260810 22:39:31.487353 28386 mvcc.cc:205] Tried to move back new op lower bound from 7317100836766928896 to 7317100836579188736. Current Snapshot: MvccSnapshot[applied={T|T < 7317100836766928896}]
I20260810 22:39:31.494906 28388 mvcc.cc:205] Tried to move back new op lower bound from 7317100836766928896 to 7317100836579188736. Current Snapshot: MvccSnapshot[applied={T|T < 7317100836766928896}]
I20260810 22:39:31.498221 27005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:31.498406 27005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:31.502636 27005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcd41744f48a4235ae0ca99f5a08bd61: Sending DeleteTablet for 3 replicas of tablet 501d921d286e4db3b591471ed5d872a4
I20260810 22:39:31.503520 28266 tablet_service.cc:1514] Processing DeleteTablet for tablet 501d921d286e4db3b591471ed5d872a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:31 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:49392
I20260810 22:39:31.504601 27974 tablet_service.cc:1514] Processing DeleteTablet for tablet 501d921d286e4db3b591471ed5d872a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:31 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:60868
I20260810 22:39:31.504665 28119 tablet_service.cc:1514] Processing DeleteTablet for tablet 501d921d286e4db3b591471ed5d872a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:31 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:33852
I20260810 22:39:31.505427 28397 tablet_replica.cc:330] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: stopping tablet replica
I20260810 22:39:31.505699 28398 tablet_replica.cc:330] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: stopping tablet replica
I20260810 22:39:31.505949 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 26971
I20260810 22:39:31.506012 28398 raft_consensus.cc:2229] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:31.508631 28398 raft_consensus.cc:2258] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:31.512653 28398 ts_tablet_manager.cc:1905] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:31.516192 28398 ts_tablet_manager.cc:1918] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:31.516299 28398 log.cc:1196] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/501d921d286e4db3b591471ed5d872a4
I20260810 22:39:31.516680 28398 ts_tablet_manager.cc:1939] T 501d921d286e4db3b591471ed5d872a4 P 8d0cb262e1124afa902aa295415cc336: Deleting consensus metadata
I20260810 22:39:31.519217 28397 raft_consensus.cc:2229] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:31.519640 28397 raft_consensus.cc:2258] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 22:39:31.520417 27925 connection.cc:441] server connection from 127.0.0.1:60868 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60868 (request call id 1) could send its response
I20260810 22:39:31.521837 28397 ts_tablet_manager.cc:1905] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:31.521890 28395 tablet_replica.cc:330] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: stopping tablet replica
I20260810 22:39:31.522439 28395 raft_consensus.cc:2229] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:31.522786 28395 raft_consensus.cc:2258] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:31.525440 28395 ts_tablet_manager.cc:1905] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:31.526168 28397 ts_tablet_manager.cc:1918] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:31.526304 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.505442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59398 (local address 127.19.202.125:42995)
0810 22:39:31.505556 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.505560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.510395 (+ 4835us) server_negotiation.cc:407] Connection header received
0810 22:39:31.510903 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.510909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.510968 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.511022 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.516891 (+ 5869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.517819 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.519148 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.519467 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.519557 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.522187 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.522191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.522194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.522222 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.524349 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.524887 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.524891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.524893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.524986 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.525312 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.525315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.525317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.525555 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.525678 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.526037 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.526155 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:39:31.526310 28397 log.cc:1196] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/501d921d286e4db3b591471ed5d872a4
I20260810 22:39:31.527017 28397 ts_tablet_manager.cc:1939] T 501d921d286e4db3b591471ed5d872a4 P be93acea2f3f4a8caad402f6374d483c: Deleting consensus metadata
W20260810 22:39:31.528730 28068 connection.cc:441] server connection from 127.0.0.1:33852 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33852 (request call id 1) could send its response
I20260810 22:39:31.529006 28395 ts_tablet_manager.cc:1918] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:31.529107 28395 log.cc:1196] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/501d921d286e4db3b591471ed5d872a4
I20260810 22:39:31.529420 28395 ts_tablet_manager.cc:1939] T 501d921d286e4db3b591471ed5d872a4 P 6361f8d43c044c939eaa4585da651f4d: Deleting consensus metadata
W20260810 22:39:31.531715 28218 connection.cc:441] server connection from 127.0.0.1:49392 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49392 (request call id 1) could send its response
I20260810 22:39:31.537376 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.504941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.19.202.124:36949)
0810 22:39:31.505053 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.505058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.527850 (+ 22792us) server_negotiation.cc:407] Connection header received
0810 22:39:31.528053 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.528056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.528106 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.528154 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.529052 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.529780 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.530945 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.531261 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.531335 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.532946 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.532950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.532952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.532989 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.535266 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.535771 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.535774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.535775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.535872 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.536238 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.536243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.536246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.536493 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.536631 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.537112 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.537228 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:31.539659 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.527783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54832 (local address 127.19.202.124:36949)
0810 22:39:31.527893 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.527897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.530016 (+ 2119us) server_negotiation.cc:407] Connection header received
0810 22:39:31.530152 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.530155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.530193 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.530238 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.532139 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.532882 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.534141 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.534525 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.534643 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.536108 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.536112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.536115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.536138 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.537920 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.538384 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.538387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.538389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.538466 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.538743 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.538749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.538753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.538994 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.539126 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.539425 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.539528 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:31.541262 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:31.527416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59414 (local address 127.19.202.125:42995)
0810 22:39:31.527518 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:31.527521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:31.529744 (+ 2223us) server_negotiation.cc:407] Connection header received
0810 22:39:31.529849 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:31.529853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:31.529899 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:31.529965 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:31.531207 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.531917 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.533102 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:31.533466 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:31.533549 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:31.535775 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:31.535778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:31.535781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:31.535801 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:31.539009 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.539961 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.539964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.539966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.540046 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:31.540452 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:31.540455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:31.540456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:31.540619 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:31.540743 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:31.540994 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 22:39:31.541095 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 22:39:32.490531 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:32.489988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:46981)
0810 22:39:32.490306 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:32.490425 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
W20260810 22:39:32.490860 28045 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:34879 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:32.491958 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:32.491499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:40811)
0810 22:39:32.491762 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:32.491854 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
W20260810 22:39:32.492314 28190 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:34879 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:32.492971 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:32.492556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:45389)
0810 22:39:32.492757 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:32.492867 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
W20260810 22:39:32.493323 28337 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:34879 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:33.003159 28411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcd41744f48a4235ae0ca99f5a08bd61)
I20260810 22:39:33.003159 28410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcd41744f48a4235ae0ca99f5a08bd61)
I20260810 22:39:33.003415 28410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:33.003408 28411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:33.004417 28410 leader_election.cc:288] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879), 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124:36949)
I20260810 22:39:33.004432 28411 leader_election.cc:288] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879), 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125:42995)
I20260810 22:39:33.005031 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.004523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39460)
0810 22:39:33.004827 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.004927 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":70,"threads_started":1}
I20260810 22:39:33.005044 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.004531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39458)
0810 22:39:33.004861 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.004940 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":75,"threads_started":1}
W20260810 22:39:33.005218 27637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
W20260810 22:39:33.005218 27838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:33.006181 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.005949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39478)
0810 22:39:33.006071 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.006115 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260810 22:39:33.006290 27838 leader_election.cc:334] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879): Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:33.006512 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.006041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39494)
0810 22:39:33.006324 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.006435 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260810 22:39:33.006614 27637 leader_election.cc:334] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879): Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:33.009022 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.004709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42995 (local address 127.0.0.1:59422)
0810 22:39:33.005380 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.005397 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:33.005509 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:33.005967 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:33.005970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:33.006002 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:33.006120 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:33.006127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.007092 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.007098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:33.007901 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:33.007910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.008620 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.008624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:33.008697 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:33.008710 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 22:39:33.008759 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 22:39:33.008889 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"spinlock_wait_cycles":4608,"thread_start_us":73,"threads_started":1}
I20260810 22:39:33.009093 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.004978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.19.202.124:36949)
0810 22:39:33.005679 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:33.005685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:33.005698 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:33.005844 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:33.005847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:33.005892 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:33.005974 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:33.006368 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.007052 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.008083 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.008461 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.008547 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:33.008621 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:33.008917 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 22:39:33.008978 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":257,"server-negotiator.queue_time_us":596,"spinlock_wait_cycles":5632,"thread_start_us":56,"threads_started":1}
I20260810 22:39:33.009058 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.004987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59422 (local address 127.19.202.125:42995)
0810 22:39:33.005258 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:33.005263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:33.005604 (+ 341us) server_negotiation.cc:407] Connection header received
0810 22:39:33.005688 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:33.005691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:33.005748 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:33.005829 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:33.006252 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.006964 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.008033 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.008488 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.008614 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:33.008762 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:33.008826 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 22:39:33.008928 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":190,"spinlock_wait_cycles":896,"thread_start_us":75,"threads_started":1}
I20260810 22:39:33.009057 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.005352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:36949 (local address 127.0.0.1:54842)
0810 22:39:33.005581 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.005595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:33.005717 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:33.006050 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:33.006052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:33.006068 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:33.006177 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:33.006184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.007154 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.007156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:33.007943 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:33.007953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:33.008568 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:33.008574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:33.008762 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:33.008814 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 22:39:33.008898 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:39:33.008950 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":48,"thread_start_us":81,"threads_started":1}
I20260810 22:39:33.009778 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bbfefb06f6142a9890814aaf9536200" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" is_pre_election: true
I20260810 22:39:33.009783 27860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6bbfefb06f6142a9890814aaf9536200" is_pre_election: true
I20260810 22:39:33.010000 27663 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbfefb06f6142a9890814aaf9536200 in term 1.
I20260810 22:39:33.009992 27860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dc88aa2aeb94b9983677db2536a7b8d in term 1.
I20260810 22:39:33.010357 27840 leader_election.cc:302] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dc88aa2aeb94b9983677db2536a7b8d, 6bbfefb06f6142a9890814aaf9536200; no voters: fcd41744f48a4235ae0ca99f5a08bd61
I20260810 22:39:33.010407 27639 leader_election.cc:302] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dc88aa2aeb94b9983677db2536a7b8d, 6bbfefb06f6142a9890814aaf9536200; no voters: fcd41744f48a4235ae0ca99f5a08bd61
I20260810 22:39:33.010601 28411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 22:39:33.010656 28411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcd41744f48a4235ae0ca99f5a08bd61)
I20260810 22:39:33.010630 28410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 22:39:33.010696 28411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:39:33.010695 28410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcd41744f48a4235ae0ca99f5a08bd61)
I20260810 22:39:33.010756 28410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:39:33.011868 28411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:33.011861 28410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:33.012152 28410 leader_election.cc:288] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [CANDIDATE]: Term 2 election: Requested vote from peers fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879), 6bbfefb06f6142a9890814aaf9536200 (127.19.202.124:36949)
I20260810 22:39:33.012234 28411 leader_election.cc:288] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 2 election: Requested vote from peers fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879), 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125:42995)
I20260810 22:39:33.012458 27860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6bbfefb06f6142a9890814aaf9536200"
I20260810 22:39:33.012581 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.012317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39498)
0810 22:39:33.012463 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.012512 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 22:39:33.012639 27860 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1dc88aa2aeb94b9983677db2536a7b8d in current term 2: Already voted for candidate 6bbfefb06f6142a9890814aaf9536200 in this term.
I20260810 22:39:33.012648 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bbfefb06f6142a9890814aaf9536200" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
I20260810 22:39:33.012820 27663 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6bbfefb06f6142a9890814aaf9536200 in current term 2: Already voted for candidate 1dc88aa2aeb94b9983677db2536a7b8d in this term.
I20260810 22:39:33.012833 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.012395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39502)
0810 22:39:33.012676 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.012758 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260810 22:39:33.013262 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.013047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39508)
0810 22:39:33.013150 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.013204 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 22:39:33.013360 27838 leader_election.cc:334] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879): Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:33.013446 27838 leader_election.cc:302] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bbfefb06f6142a9890814aaf9536200; no voters: 1dc88aa2aeb94b9983677db2536a7b8d, fcd41744f48a4235ae0ca99f5a08bd61
I20260810 22:39:33.013496 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.013244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39522)
0810 22:39:33.013376 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.013430 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 22:39:33.013577 27637 leader_election.cc:334] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879): Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:33.013594 28411 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 22:39:33.013635 27637 leader_election.cc:302] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dc88aa2aeb94b9983677db2536a7b8d; no voters: 6bbfefb06f6142a9890814aaf9536200, fcd41744f48a4235ae0ca99f5a08bd61
I20260810 22:39:33.013761 28410 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 22:39:33.491856 28421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.491445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:47085)
0810 22:39:33.491714 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.491768 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":117,"threads_started":1}
I20260810 22:39:33.493280 28422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.492907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:60217)
0810 22:39:33.493130 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.493193 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":70,"threads_started":1}
I20260810 22:39:33.494297 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:33.493822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:52855)
0810 22:39:33.494130 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:33.494201 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":74,"threads_started":1}
I20260810 22:39:34.493098 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.492609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:43777)
0810 22:39:34.492891 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.492957 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
I20260810 22:39:34.494338 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.493907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:54073)
0810 22:39:34.494212 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.494263 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":82,"threads_started":1}
I20260810 22:39:34.495391 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.495013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:57557)
0810 22:39:34.495229 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.495289 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260810 22:39:34.546073 28427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:34.546236 28427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:34.546717 28427 leader_election.cc:288] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879), 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125:42995)
I20260810 22:39:34.547235 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bbfefb06f6142a9890814aaf9536200" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" is_pre_election: true
I20260810 22:39:34.547406 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.546938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39732)
0810 22:39:34.547185 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.547276 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":92,"threads_started":1}
I20260810 22:39:34.547423 27663 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbfefb06f6142a9890814aaf9536200 in term 2.
I20260810 22:39:34.547739 27840 leader_election.cc:302] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dc88aa2aeb94b9983677db2536a7b8d, 6bbfefb06f6142a9890814aaf9536200; no voters:
I20260810 22:39:34.547950 28427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 22:39:34.548004 28427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:34.548045 28427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 2 FOLLOWER]: Advancing to term 3
I20260810 22:39:34.548180 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.547950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39746)
0810 22:39:34.548075 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.548127 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 22:39:34.548277 27838 leader_election.cc:334] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879): Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:34.549217 28427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:34.549559 28427 leader_election.cc:288] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 3 election: Requested vote from peers fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879), 1dc88aa2aeb94b9983677db2536a7b8d (127.19.202.125:42995)
I20260810 22:39:34.549820 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.549624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39750)
0810 22:39:34.549719 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.549770 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 22:39:34.549932 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bbfefb06f6142a9890814aaf9536200" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1dc88aa2aeb94b9983677db2536a7b8d"
I20260810 22:39:34.550060 27663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 2 FOLLOWER]: Advancing to term 3
I20260810 22:39:34.550468 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.550269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39766)
0810 22:39:34.550366 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.550409 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:39:34.550632 27838 leader_election.cc:334] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879): Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
I20260810 22:39:34.551113 27663 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbfefb06f6142a9890814aaf9536200 in term 3.
I20260810 22:39:34.551383 27840 leader_election.cc:302] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dc88aa2aeb94b9983677db2536a7b8d, 6bbfefb06f6142a9890814aaf9536200; no voters: fcd41744f48a4235ae0ca99f5a08bd61
I20260810 22:39:34.551569 28427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 22:39:34.551746 28427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [term 3 LEADER]: Becoming Leader. State: Replica: 6bbfefb06f6142a9890814aaf9536200, State: Running, Role: LEADER
I20260810 22:39:34.551998 28427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } }
I20260810 22:39:34.553380 28430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bbfefb06f6142a9890814aaf9536200. Latest consensus state: current_term: 3 leader_uuid: "6bbfefb06f6142a9890814aaf9536200" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:34.553514 28430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:34.553774 28432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:34.556208 28432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:34.556600 28432 catalog_manager.cc:1259] Loaded cluster ID: 3ccf87bfb5c24f359435f4cd4289174a
I20260810 22:39:34.556664 28432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:34.557296 28432 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:34.557687 28432 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Loaded TSK: 0
I20260810 22:39:34.558349 28432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:39:34.582209 27847 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:34.585176 27847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:34.587832 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.587619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39778)
0810 22:39:34.587736 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.587778 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 22:39:34.587950 27663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [term 3 FOLLOWER]: Refusing update from remote peer 6bbfefb06f6142a9890814aaf9536200: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 22:39:34.588480 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.588252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.0.0.1:39792)
0810 22:39:34.588355 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.588406 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 22:39:34.588400 28430 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 22:39:34.588609 27838 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 -> Peer fcd41744f48a4235ae0ca99f5a08bd61 (127.19.202.126:34879): Couldn't send request to peer fcd41744f48a4235ae0ca99f5a08bd61. Status: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 22:39:34.589756 28434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bbfefb06f6142a9890814aaf9536200. Latest consensus state: current_term: 3 leader_uuid: "6bbfefb06f6142a9890814aaf9536200" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:34.589854 28434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:39:34.590895 28430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6bbfefb06f6142a9890814aaf9536200" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:34.591115 28430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:34.591255 28434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6bbfefb06f6142a9890814aaf9536200" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fcd41744f48a4235ae0ca99f5a08bd61" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34879 } } peers { permanent_uuid: "1dc88aa2aeb94b9983677db2536a7b8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42995 } attrs { promote: false } } peers { permanent_uuid: "6bbfefb06f6142a9890814aaf9536200" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 36949 } attrs { promote: false } } }
I20260810 22:39:34.591387 28434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc88aa2aeb94b9983677db2536a7b8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:39:34.601464 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.596814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39199 (local address 127.0.0.1:49408)
0810 22:39:34.596972 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.596993 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:34.597069 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:34.597870 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:34.597873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:34.597890 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:34.598003 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:34.598009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.599048 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.599051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:34.600175 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:34.600184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.601074 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.601077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:34.601173 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:34.601194 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 22:39:34.601283 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 22:39:34.601353 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 22:39:34.601508 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.596950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.19.202.67:39199)
0810 22:39:34.597496 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:34.597502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:34.597517 (+ 15us) server_negotiation.cc:407] Connection header received
0810 22:39:34.597570 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:34.597573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:34.597646 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:34.597736 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:34.598113 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.598941 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.600313 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.600948 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.601063 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:34.601125 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:34.601299 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 22:39:34.601371 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":73,"threads_started":1}
I20260810 22:39:34.602312 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.597248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:40455 (local address 127.0.0.1:33870)
0810 22:39:34.597934 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.597946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:34.598021 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:34.598673 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:34.598676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:34.598692 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:34.598786 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:34.598792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.599620 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.599626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:34.601088 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:34.601100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.601987 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.601990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:34.602083 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:34.602105 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:39:34.602170 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 22:39:34.602220 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":135,"thread_start_us":92,"threads_started":1}
I20260810 22:39:34.602300 28266 tablet_service.cc:1467] Processing CreateTablet for tablet 0a82548f3995433fb262196d2c9bfb14 (DEFAULT_TABLE table=test-table [id=2048622c4c5449e6ac0d33fba501120e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:34.602615 28266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a82548f3995433fb262196d2c9bfb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:34.602844 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.597349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33870 (local address 127.19.202.66:40455)
0810 22:39:34.598450 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:34.598476 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:34.598487 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:39:34.598530 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:34.598532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:34.598567 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:34.598665 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:34.598888 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.599503 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.601232 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.601828 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.602529 (+ 701us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:34.602601 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:34.602631 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 22:39:34.602689 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":88,"threads_started":1}
I20260810 22:39:34.603308 28119 tablet_service.cc:1467] Processing CreateTablet for tablet 0a82548f3995433fb262196d2c9bfb14 (DEFAULT_TABLE table=test-table [id=2048622c4c5449e6ac0d33fba501120e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:34.603564 28119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a82548f3995433fb262196d2c9bfb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:34.605140 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.597555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44641 (local address 127.0.0.1:60882)
0810 22:39:34.598190 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.598199 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:34.598283 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:34.598609 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:34.598612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:34.598628 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:34.598716 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:34.598722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.599682 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.599685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:34.601094 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:34.601100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.601940 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.601944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:34.602027 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:34.602052 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:39:34.604944 (+ 2892us) client_negotiation.cc:240] Negotiation successful
0810 22:39:34.605020 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":97,"thread_start_us":98,"threads_started":1}
I20260810 22:39:34.605170 28445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.597889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60882 (local address 127.19.202.65:44641)
0810 22:39:34.598109 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:34.598114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:34.598282 (+ 168us) server_negotiation.cc:407] Connection header received
0810 22:39:34.598354 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:34.598357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:34.598407 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:34.598501 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:34.598822 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.599591 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.601232 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.601830 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.601952 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:34.602012 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:34.604964 (+ 2952us) server_negotiation.cc:299] Negotiation successful
0810 22:39:34.605043 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260810 22:39:34.605716 27974 tablet_service.cc:1467] Processing CreateTablet for tablet 0a82548f3995433fb262196d2c9bfb14 (DEFAULT_TABLE table=test-table [id=2048622c4c5449e6ac0d33fba501120e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:34.605983 27974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a82548f3995433fb262196d2c9bfb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:34.606187 28447 tablet_bootstrap.cc:492] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: Bootstrap starting.
I20260810 22:39:34.607085 28446 tablet_bootstrap.cc:492] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: Bootstrap starting.
I20260810 22:39:34.607136 28447 tablet_bootstrap.cc:654] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:34.608052 28446 tablet_bootstrap.cc:654] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:34.608429 28447 tablet_bootstrap.cc:492] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: No bootstrap required, opened a new log
I20260810 22:39:34.608644 28449 tablet_bootstrap.cc:492] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336: Bootstrap starting.
I20260810 22:39:34.608644 28447 ts_tablet_manager.cc:1397] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:34.609081 28447 raft_consensus.cc:348] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.609210 28447 raft_consensus.cc:374] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:34.609256 28447 raft_consensus.cc:729] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93acea2f3f4a8caad402f6374d483c, State: Initialized, Role: FOLLOWER
I20260810 22:39:34.609390 28446 tablet_bootstrap.cc:492] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: No bootstrap required, opened a new log
I20260810 22:39:34.609483 28446 ts_tablet_manager.cc:1397] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:34.609544 28447 consensus_queue.cc:260] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.609797 28449 tablet_bootstrap.cc:654] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:34.609705 28451 raft_consensus.cc:482] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:34.609973 28451 raft_consensus.cc:504] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.609963 28446 raft_consensus.cc:348] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.610067 28446 raft_consensus.cc:374] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:34.610106 28446 raft_consensus.cc:729] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6361f8d43c044c939eaa4585da651f4d, State: Initialized, Role: FOLLOWER
I20260810 22:39:34.610067 28447 ts_tablet_manager.cc:1428] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:34.610244 28446 consensus_queue.cc:260] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.610605 28446 ts_tablet_manager.cc:1428] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:34.611372 28287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a82548f3995433fb262196d2c9bfb14" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6361f8d43c044c939eaa4585da651f4d" is_pre_election: true
I20260810 22:39:34.611503 28287 raft_consensus.cc:2454] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93acea2f3f4a8caad402f6374d483c in term 0.
I20260810 22:39:34.611864 28068 leader_election.cc:302] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6361f8d43c044c939eaa4585da651f4d, be93acea2f3f4a8caad402f6374d483c; no voters:
I20260810 22:39:34.612162 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.611518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:49907)
0810 22:39:34.611993 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.612055 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375}
I20260810 22:39:34.612262 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.611798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:38837)
0810 22:39:34.612154 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.612200 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291}
I20260810 22:39:34.612314 27994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a82548f3995433fb262196d2c9bfb14" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d0cb262e1124afa902aa295415cc336" is_pre_election: true
I20260810 22:39:34.612071 28451 leader_election.cc:288] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199), 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641)
W20260810 22:39:34.612911 28072 leader_election.cc:341] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641): Illegal state: must be running to vote when last-logged opid is not known
I20260810 22:39:34.613433 28452 raft_consensus.cc:2781] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:34.613533 28452 raft_consensus.cc:482] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:34.613585 28452 raft_consensus.cc:3037] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:34.613686 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.613337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:59301)
0810 22:39:34.613559 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.613616 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260810 22:39:34.614075 28189 heartbeater.cc:503] Master 127.19.202.124:36949 was elected leader, sending a full tablet report...
I20260810 22:39:34.614189 28336 heartbeater.cc:503] Master 127.19.202.124:36949 was elected leader, sending a full tablet report...
I20260810 22:39:34.615134 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.614889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:51667)
0810 22:39:34.615033 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.615083 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 22:39:34.615381 28449 tablet_bootstrap.cc:492] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336: No bootstrap required, opened a new log
I20260810 22:39:34.615494 28449 ts_tablet_manager.cc:1397] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260810 22:39:34.616005 28449 raft_consensus.cc:348] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.616092 28449 raft_consensus.cc:374] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:34.616114 28449 raft_consensus.cc:729] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d0cb262e1124afa902aa295415cc336, State: Initialized, Role: FOLLOWER
I20260810 22:39:34.616065 28452 raft_consensus.cc:504] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.616257 28449 consensus_queue.cc:260] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.616729 28287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a82548f3995433fb262196d2c9bfb14" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6361f8d43c044c939eaa4585da651f4d"
I20260810 22:39:34.616864 28287 raft_consensus.cc:3037] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:34.617091 28449 ts_tablet_manager.cc:1428] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:34.618084 28287 raft_consensus.cc:2454] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93acea2f3f4a8caad402f6374d483c in term 1.
I20260810 22:39:34.618225 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.617911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:49861)
0810 22:39:34.618081 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.618141 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 22:39:34.618404 28068 leader_election.cc:302] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6361f8d43c044c939eaa4585da651f4d, be93acea2f3f4a8caad402f6374d483c; no voters:
I20260810 22:39:34.618574 28455 raft_consensus.cc:2781] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:34.618690 28455 raft_consensus.cc:686] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 1 LEADER]: Becoming Leader. State: Replica: be93acea2f3f4a8caad402f6374d483c, State: Running, Role: LEADER
I20260810 22:39:34.618826 28455 consensus_queue.cc:237] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } }
I20260810 22:39:34.618937 27994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a82548f3995433fb262196d2c9bfb14" candidate_uuid: "be93acea2f3f4a8caad402f6374d483c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d0cb262e1124afa902aa295415cc336"
I20260810 22:39:34.619056 27994 raft_consensus.cc:3037] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:34.619267 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.619031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:49421)
0810 22:39:34.619164 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.619211 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 22:39:34.620035 27994 raft_consensus.cc:2454] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93acea2f3f4a8caad402f6374d483c in term 1.
I20260810 22:39:34.620426 28452 leader_election.cc:288] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [CANDIDATE]: Term 1 election: Requested vote from peers 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199), 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641)
I20260810 22:39:34.620642 28044 heartbeater.cc:503] Master 127.19.202.124:36949 was elected leader, sending a full tablet report...
I20260810 22:39:34.620867 27849 catalog_manager.cc:5495] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c reported cstate change: term changed from 0 to 1, leader changed from <none> to be93acea2f3f4a8caad402f6374d483c (127.19.202.66). New cstate: current_term: 1 leader_uuid: "be93acea2f3f4a8caad402f6374d483c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93acea2f3f4a8caad402f6374d483c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 40455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:34.621487 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.621241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:43403)
0810 22:39:34.621358 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.621422 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 22:39:34.633227 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.630667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.19.202.66:40455)
0810 22:39:34.630777 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:34.630781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:34.630792 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:39:34.630878 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:34.630880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:34.630923 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:34.630983 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:34.631182 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.631651 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.632398 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:34.632593 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:34.632664 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:34.632909 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:34.632977 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:34.633025 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:34.633105 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 22:39:34.633143 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:39:34.635040 27994 raft_consensus.cc:1261] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [term 1 FOLLOWER]: Refusing update from remote peer be93acea2f3f4a8caad402f6374d483c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:34.635044 28287 raft_consensus.cc:1261] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Refusing update from remote peer be93acea2f3f4a8caad402f6374d483c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:34.635497 28452 consensus_queue.cc:1035] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6361f8d43c044c939eaa4585da651f4d" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:34.635661 28455 consensus_queue.cc:1035] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d0cb262e1124afa902aa295415cc336" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:34.637306 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.637045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:43123)
0810 22:39:34.637191 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.637244 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 22:39:34.637742 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.637523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:35519)
0810 22:39:34.637615 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.637673 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 22:39:34.640400 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.640168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.66:46337)
0810 22:39:34.640295 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.640344 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 22:39:34.640822 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.640503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.65:37967)
0810 22:39:34.640686 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.640736 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260810 22:39:34.641714 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.641439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34879 (local address 127.19.202.67:35589)
0810 22:39:34.641606 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.641655 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260810 22:39:34.643785 27847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:34.643944 27847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:34.647580 27847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bbfefb06f6142a9890814aaf9536200: Sending DeleteTablet for 3 replicas of tablet 0a82548f3995433fb262196d2c9bfb14
I20260810 22:39:34.648193 28119 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a82548f3995433fb262196d2c9bfb14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:34 UTC) from {username='oryx', principal='oryx/127.19.202.124@KRBTEST.COM'} at 127.0.0.1:33870
I20260810 22:39:34.648202 28266 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a82548f3995433fb262196d2c9bfb14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:34 UTC) from {username='oryx', principal='oryx/127.19.202.124@KRBTEST.COM'} at 127.0.0.1:49408
I20260810 22:39:34.648516 28471 tablet_replica.cc:330] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: stopping tablet replica
I20260810 22:39:34.648517 28473 tablet_replica.cc:330] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: stopping tablet replica
I20260810 22:39:34.648658 28471 raft_consensus.cc:2229] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:34.648653 27974 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a82548f3995433fb262196d2c9bfb14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:34 UTC) from {username='oryx', principal='oryx/127.19.202.124@KRBTEST.COM'} at 127.0.0.1:60882
I20260810 22:39:34.648681 28473 raft_consensus.cc:2229] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:34.648830 28473 raft_consensus.cc:2258] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:34.648891 28471 raft_consensus.cc:2258] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:34.649173 28474 tablet_replica.cc:330] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336: stopping tablet replica
I20260810 22:39:34.649326 28474 raft_consensus.cc:2229] T 0a82548f3995433fb262196d2c9bfb14 P 8d0cb262e1124afa902aa295415cc336 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:34.649358 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 27815
I20260810 22:39:34.653226 28473 ts_tablet_manager.cc:1905] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:34.653632 28471 ts_tablet_manager.cc:1905] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:34.655278 28473 ts_tablet_manager.cc:1918] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:34.655376 28473 log.cc:1196] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/0a82548f3995433fb262196d2c9bfb14
I20260810 22:39:34.655643 28473 ts_tablet_manager.cc:1939] T 0a82548f3995433fb262196d2c9bfb14 P 6361f8d43c044c939eaa4585da651f4d: Deleting consensus metadata
I20260810 22:39:34.656213 27837 catalog_manager.cc:4841] TS 6361f8d43c044c939eaa4585da651f4d (127.19.202.67:39199): tablet 0a82548f3995433fb262196d2c9bfb14 (table test-table [id=2048622c4c5449e6ac0d33fba501120e]) successfully deleted
I20260810 22:39:34.656895 28471 ts_tablet_manager.cc:1918] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:34.656989 28471 log.cc:1196] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/0a82548f3995433fb262196d2c9bfb14
I20260810 22:39:34.657284 28471 ts_tablet_manager.cc:1939] T 0a82548f3995433fb262196d2c9bfb14 P be93acea2f3f4a8caad402f6374d483c: Deleting consensus metadata
I20260810 22:39:34.657963 27840 catalog_manager.cc:4841] TS be93acea2f3f4a8caad402f6374d483c (127.19.202.66:40455): tablet 0a82548f3995433fb262196d2c9bfb14 (table test-table [id=2048622c4c5449e6ac0d33fba501120e]) successfully deleted
I20260810 22:39:34.660187 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:34.659941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44641 (local address 127.0.0.1:60888)
0810 22:39:34.660070 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:34.660117 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:44641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 22:39:34.660197 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 28052
W20260810 22:39:34.663071 27840 catalog_manager.cc:4584] TS 8d0cb262e1124afa902aa295415cc336 (127.19.202.65:44641): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a82548f3995433fb262196d2c9bfb14: Network error: Client connection negotiation failed: client connection to 127.19.202.65:44641: connect: Connection refused (error 111)
I20260810 22:39:34.669436 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 28196
I20260810 22:39:34.676271 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 27505
I20260810 22:39:34.682837 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 27623
2026-08-10T22:39:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6794 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:34 dist-test-slave-1x4g krb5kdc[28478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:34 dist-test-slave-1x4g krb5kdc[28478](info): set up 2 sockets
Aug 10 22:39:34 dist-test-slave-1x4g krb5kdc[28478](info): commencing operation
krb5kdc: starting...
W20260810 22:39:35.730610 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:35 dist-test-slave-1x4g krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab.
Entry for principal oryx/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab.
I20260810 22:39:35.847349 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:32773
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab
--principal=oryx/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:32773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:35.952200 28492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:35.952441 28492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:35.952478 28492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:35.956300 28492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:35.956346 28492 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 22:39:35.956363 28492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:35.956380 28492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:35.956395 28492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:35.956409 28492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:35.961311 28492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:32773
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.126.keytab
--principal=oryx/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:32773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:35.962251 28492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:35.963126 28492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:35.971431 28497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:35.971415 28501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:35.971489 28492 server_base.cc:1034] running on GCE node
W20260810 22:39:35.971428 28499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:35.972136 28492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:35.973219 28492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:35.974399 28492 hybrid_clock.cc:648] HybridClock initialized: now 1786401575974371 us; error 43 us; skew 500 ppm
Aug 10 22:39:35 dist-test-slave-1x4g krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401575, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:35.981284 28492 init.cc:377] Logged in from keytab as oryx/127.19.202.126@KRBTEST.COM (short username oryx)
I20260810 22:39:35.982077 28492 webserver.cc:460] Webserver started at http://127.19.202.126:45391/ using document root <none> and password file <none>
I20260810 22:39:35.982549 28492 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:35.982602 28492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:35.982748 28492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:35.984618 28492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "6c72f50ae93a4020b8352fdabff2eb76"
format_stamp: "Formatted at 2026-08-10 22:39:35 on dist-test-slave-1x4g"
server_key: "9ab6505b1509e34c89c65cbef48c0536"
server_key_iv: "b2e7436e78022be93ec4b410fff0585f"
server_key_version: "encryptionkey@0"
I20260810 22:39:35.985229 28492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "6c72f50ae93a4020b8352fdabff2eb76"
format_stamp: "Formatted at 2026-08-10 22:39:35 on dist-test-slave-1x4g"
server_key: "9ab6505b1509e34c89c65cbef48c0536"
server_key_iv: "b2e7436e78022be93ec4b410fff0585f"
server_key_version: "encryptionkey@0"
I20260810 22:39:35.988530 28492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:35.990821 28510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:35.991988 28492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:35.992113 28492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "6c72f50ae93a4020b8352fdabff2eb76"
format_stamp: "Formatted at 2026-08-10 22:39:35 on dist-test-slave-1x4g"
server_key: "9ab6505b1509e34c89c65cbef48c0536"
server_key_iv: "b2e7436e78022be93ec4b410fff0585f"
server_key_version: "encryptionkey@0"
I20260810 22:39:35.992220 28492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:36.002170 28492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:36.004730 28492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:36.004997 28492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:36.012872 28492 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:32773
I20260810 22:39:36.012895 28575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:32773 every 8 connection(s)
I20260810 22:39:36.013751 28492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:36.016352 28577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:36.021665 28577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Bootstrap starting.
I20260810 22:39:36.022535 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 28492
I20260810 22:39:36.022678 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:36.023008 20265 external_mini_cluster.cc:1428] Setting key b09c7a713f23c966a3ec7694dea62f1c
I20260810 22:39:36.024540 28577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:36.025334 28577 log.cc:824] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:36.027103 28577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: No bootstrap required, opened a new log
I20260810 22:39:36.030488 28577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f50ae93a4020b8352fdabff2eb76" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 32773 } }
I20260810 22:39:36.030694 28577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:36.030773 28577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c72f50ae93a4020b8352fdabff2eb76, State: Initialized, Role: FOLLOWER
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:36.031180 28577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f50ae93a4020b8352fdabff2eb76" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 32773 } }
I20260810 22:39:36.031356 28577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:36.031464 28577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:36.031581 28577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:36.032814 28577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f50ae93a4020b8352fdabff2eb76" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 32773 } }
I20260810 22:39:36.033231 28577 leader_election.cc:302] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c72f50ae93a4020b8352fdabff2eb76; no voters:
I20260810 22:39:36.033593 28577 leader_election.cc:288] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:36.033696 28582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:36.033905 28582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [term 1 LEADER]: Becoming Leader. State: Replica: 6c72f50ae93a4020b8352fdabff2eb76, State: Running, Role: LEADER
I20260810 22:39:36.034300 28582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f50ae93a4020b8352fdabff2eb76" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 32773 } }
I20260810 22:39:36.034878 28577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:36.036098 28584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c72f50ae93a4020b8352fdabff2eb76. Latest consensus state: current_term: 1 leader_uuid: "6c72f50ae93a4020b8352fdabff2eb76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f50ae93a4020b8352fdabff2eb76" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 32773 } } }
I20260810 22:39:36.036095 28583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c72f50ae93a4020b8352fdabff2eb76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f50ae93a4020b8352fdabff2eb76" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 32773 } } }
I20260810 22:39:36.036250 28583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:36.036250 28584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:36.036857 28591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:36.037901 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.024046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.19.202.126:32773)
0810 22:39:36.024509 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.024522 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.024556 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:36.025279 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.025297 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.025669 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.025982 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.026776 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.027752 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.028595 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.029085 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.029249 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.031797 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.031805 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.031821 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.031853 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.034077 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.034601 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.034607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.034613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.034753 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.035070 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.035074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.035075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.035605 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.035797 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.036601 (+ 804us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.036981 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":190,"threads_started":1}
I20260810 22:39:36.039942 28591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:36.044863 28591 catalog_manager.cc:1347] Generated new cluster ID: 7ab2411207bb43189eadbffc54e01f74
I20260810 22:39:36.044929 28591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:36.063705 28591 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:36.064590 28591 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:36.070101 28591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Generated new TSK 0
I20260810 22:39:36.070822 28591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab.
Entry for principal oryx/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab.
I20260810 22:39:36.107373 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:32773
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab
--principal=oryx/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:36.215415 28603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:36.215692 28603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:36.215766 28603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:36.219683 28603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:36.219767 28603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:36.219838 28603 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:36.219924 28603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:36.224722 28603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab
--principal=oryx/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:32773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:36.225751 28603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:36.226734 28603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:36.236019 28609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:36.236142 28612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:36.236052 28610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:36.236325 28603 server_base.cc:1034] running on GCE node
I20260810 22:39:36.236867 28603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:36.237522 28603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:36.238711 28603 hybrid_clock.cc:648] HybridClock initialized: now 1786401576238683 us; error 42 us; skew 500 ppm
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:36.241508 28603 init.cc:377] Logged in from keytab as oryx/127.19.202.65@KRBTEST.COM (short username oryx)
I20260810 22:39:36.242398 28603 webserver.cc:460] Webserver started at http://127.19.202.65:40653/ using document root <none> and password file <none>
I20260810 22:39:36.242967 28603 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:36.243064 28603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:36.243310 28603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:36.245443 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "2757e381fe984fcdb38b34b7c2e7c685"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "380cbe09702ac0f58d969540e3a2f5c8"
server_key_iv: "59d1c3b4be4ba1adc801bbb6717f8c21"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.246053 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "2757e381fe984fcdb38b34b7c2e7c685"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "380cbe09702ac0f58d969540e3a2f5c8"
server_key_iv: "59d1c3b4be4ba1adc801bbb6717f8c21"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.249727 28603 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:36.252193 28619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:36.253460 28603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:36.253602 28603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "2757e381fe984fcdb38b34b7c2e7c685"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "380cbe09702ac0f58d969540e3a2f5c8"
server_key_iv: "59d1c3b4be4ba1adc801bbb6717f8c21"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.253724 28603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:36.272681 28603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:36.275553 28603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:36.275781 28603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:36.277401 28603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:36.277493 28603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:36.277554 28603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:36.277590 28603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:36.289629 28603 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:46003
I20260810 22:39:36.289697 28740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:46003 every 8 connection(s)
I20260810 22:39:36.290511 28603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:36.291306 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.278038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:57913 (local address 127.19.202.126:32773)
0810 22:39:36.278160 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.278171 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.279136 (+ 965us) server_negotiation.cc:407] Connection header received
0810 22:39:36.280081 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.280085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.280146 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.280229 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.281502 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.282073 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.283136 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.283530 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.283644 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.287381 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.287384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.287387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.287411 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.289200 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.289860 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.289868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.289869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.289945 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.290277 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.290282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.290283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.290471 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.290578 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.291031 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.291147 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:36.292110 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.278514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:32773 (local address 127.19.202.65:57913)
0810 22:39:36.278989 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.279029 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.279840 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.280407 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.280416 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.280898 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:36.281350 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.281362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.282182 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.282186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.283020 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.283028 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.283696 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.283701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.283940 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.284580 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:36.284606 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:36.287217 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:36.289359 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.289364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.289376 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.289739 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.290040 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.290046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.290049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.290180 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.290577 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:36.290584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:36.290809 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.290990 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.291265 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":142,"threads_started":1}
I20260810 22:39:36.293460 28741 heartbeater.cc:347] Connected to a master server at 127.19.202.126:32773
I20260810 22:39:36.293643 28741 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:36.293846 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 28603
I20260810 22:39:36.293972 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:36.294207 28741 heartbeater.cc:511] Master 127.19.202.126:32773 requested a full tablet report, sending...
I20260810 22:39:36.294384 20265 external_mini_cluster.cc:1428] Setting key 122694235a00eadfa7bcbf6ac988dfe2
I20260810 22:39:36.296005 28526 ts_manager.cc:194] Registered new tserver with Master: 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003)
I20260810 22:39:36.297626 28526 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:57913
I20260810 22:39:36.303306 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.295725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:34735 (local address 127.19.202.126:32773)
0810 22:39:36.295825 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.295829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.296303 (+ 474us) server_negotiation.cc:407] Connection header received
0810 22:39:36.296452 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.296455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.296500 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.296599 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.297114 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.297615 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.298805 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.299010 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.299089 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.300210 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.300213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.300215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.300234 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.301635 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.302132 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.302134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.302135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.302217 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.302550 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.302553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.302556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.302734 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.302808 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.303123 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.303207 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:39:36.303476 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.295654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:32773 (local address 127.19.202.65:34735)
0810 22:39:36.296191 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.296246 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.296343 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.296616 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.296619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.296888 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:36.296999 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.297004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.297772 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.297777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.298685 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.298696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.299126 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.299129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.299213 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.299633 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:36.299647 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:36.300069 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:36.301751 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.301757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.301762 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.302040 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.302320 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.302326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.302331 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.302440 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.302843 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:36.302849 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:36.302957 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.303075 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.303234 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for oryx/127.19.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab.
Entry for principal oryx/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab.
I20260810 22:39:36.323225 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:32773
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab
--principal=oryx/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:36.430920 28750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:36.431128 28750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:36.431165 28750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:36.435204 28750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:36.435254 28750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:36.435319 28750 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:36.435381 28750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:36.440344 28750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab
--principal=oryx/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:32773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:36.441287 28750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:36.442124 28750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:36.451012 28758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:36.451025 28755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:36.451179 28750 server_base.cc:1034] running on GCE node
W20260810 22:39:36.451072 28756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:36.451658 28750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:36.452282 28750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:36.453456 28750 hybrid_clock.cc:648] HybridClock initialized: now 1786401576453443 us; error 23 us; skew 500 ppm
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:36.456082 28750 init.cc:377] Logged in from keytab as oryx/127.19.202.66@KRBTEST.COM (short username oryx)
I20260810 22:39:36.456951 28750 webserver.cc:460] Webserver started at http://127.19.202.66:42851/ using document root <none> and password file <none>
I20260810 22:39:36.457479 28750 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:36.457571 28750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:36.457823 28750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:36.459772 28750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "30ad79205310480095187ae07559510c"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "a15d9de57cf2199dfa4098b7c4138f01"
server_key_iv: "3be94df1b0c3852a5af4e313b5de3278"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.460326 28750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "30ad79205310480095187ae07559510c"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "a15d9de57cf2199dfa4098b7c4138f01"
server_key_iv: "3be94df1b0c3852a5af4e313b5de3278"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.465257 28750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 22:39:36.467551 28765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:36.468676 28750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:36.468851 28750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "30ad79205310480095187ae07559510c"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "a15d9de57cf2199dfa4098b7c4138f01"
server_key_iv: "3be94df1b0c3852a5af4e313b5de3278"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.468978 28750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:36.490936 28750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:36.493743 28750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:36.493978 28750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:36.495401 28750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:36.495484 28750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:36.495553 28750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:36.495599 28750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:36.506702 28750 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:41341
I20260810 22:39:36.506789 28887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:41341 every 8 connection(s)
I20260810 22:39:36.507761 28750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:36.509256 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 28750
I20260810 22:39:36.509397 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:36.509680 20265 external_mini_cluster.cc:1428] Setting key 8b77b7cf56d833b7d06ab29dee39a52b
I20260810 22:39:36.509860 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.496047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:34563 (local address 127.19.202.126:32773)
0810 22:39:36.496190 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.496195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.497147 (+ 952us) server_negotiation.cc:407] Connection header received
0810 22:39:36.498059 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.498063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.498129 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.498234 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.499729 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.500477 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.501823 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.502072 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.502187 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.505818 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.505821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.505822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.505845 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.507490 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.508121 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.508127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.508132 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.508232 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.508657 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.508660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.508662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.508940 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.509020 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.509564 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.509704 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:39:36.510703 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.496453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:32773 (local address 127.19.202.66:34563)
0810 22:39:36.496985 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.497034 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.497850 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.498515 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.498527 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.499025 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:36.499586 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.499599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.500608 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.500612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.501698 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.501708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.502236 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.502240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.502404 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.502952 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:36.502978 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:36.505638 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:36.507614 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.507618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.507643 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.507972 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.508351 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.508354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.508356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.508542 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.509042 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:36.509046 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:36.509249 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.509485 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.509776 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":144,"threads_started":1}
I20260810 22:39:36.512039 28888 heartbeater.cc:347] Connected to a master server at 127.19.202.126:32773
I20260810 22:39:36.512161 28888 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:36.512724 28888 heartbeater.cc:511] Master 127.19.202.126:32773 requested a full tablet report, sending...
I20260810 22:39:36.513839 28527 ts_manager.cc:194] Registered new tserver with Master: 30ad79205310480095187ae07559510c (127.19.202.66:41341)
I20260810 22:39:36.514704 28527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:34563
WARNING: no policy specified for oryx/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:39:36.521390 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.514508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:32773 (local address 127.19.202.66:52911)
0810 22:39:36.514818 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.514834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.514934 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.515160 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.515163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.515344 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:36.515440 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.515445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.516164 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.516167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.516738 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.516746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.517305 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.517309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.517404 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.517934 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:36.517945 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:36.518219 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:36.519843 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.519846 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.519848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.520059 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.520343 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.520346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.520348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.520433 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.520871 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:36.520874 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:36.520966 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.521089 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.521222 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":109,"threads_started":1}
I20260810 22:39:36.521385 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.514567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:52911 (local address 127.19.202.126:32773)
0810 22:39:36.514659 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.514663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.514898 (+ 235us) server_negotiation.cc:407] Connection header received
0810 22:39:36.515017 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.515020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.515066 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.515138 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.515571 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.516042 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.516950 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.517181 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.517291 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.518343 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.518346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.518349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.518401 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.519728 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.520165 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.520169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.520170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.520252 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.520523 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.520527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.520529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.520731 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.520875 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.521142 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.521245 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab.
Entry for principal oryx/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab.
I20260810 22:39:36.539116 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:32773
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab
--principal=oryx/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:36.647369 28897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:36.647658 28897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:36.647718 28897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:36.651249 28897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:36.651296 28897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:36.651329 28897 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:39:36.651358 28897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:36.656193 28897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab
--principal=oryx/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.202.126:32773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:36.657200 28897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:36.658133 28897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:36.667373 28903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:36.667445 28902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:36.667387 28905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:36.667618 28897 server_base.cc:1034] running on GCE node
I20260810 22:39:36.668090 28897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:36.668687 28897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:36.669894 28897 hybrid_clock.cc:648] HybridClock initialized: now 1786401576669880 us; error 39 us; skew 500 ppm
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:36.672760 28897 init.cc:377] Logged in from keytab as oryx/127.19.202.67@KRBTEST.COM (short username oryx)
I20260810 22:39:36.673621 28897 webserver.cc:460] Webserver started at http://127.19.202.67:38791/ using document root <none> and password file <none>
I20260810 22:39:36.674171 28897 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:36.674268 28897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:36.674504 28897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:36.676467 28897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "ed981bc3e6284f2b9605fb26fbef0a03"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "2d1c1d3bb8fc8018a9f2178ff65c94bb"
server_key_iv: "2de6de95bb06e8697e0c8aa52a1f8e9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.677083 28897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "ed981bc3e6284f2b9605fb26fbef0a03"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "2d1c1d3bb8fc8018a9f2178ff65c94bb"
server_key_iv: "2de6de95bb06e8697e0c8aa52a1f8e9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.680594 28897 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:39:36.682858 28912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:36.683987 28897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:36.684126 28897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "ed981bc3e6284f2b9605fb26fbef0a03"
format_stamp: "Formatted at 2026-08-10 22:39:36 on dist-test-slave-1x4g"
server_key: "2d1c1d3bb8fc8018a9f2178ff65c94bb"
server_key_iv: "2de6de95bb06e8697e0c8aa52a1f8e9e"
server_key_version: "encryptionkey@0"
I20260810 22:39:36.684228 28897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:36.697672 28897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:36.700296 28897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:36.700501 28897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:36.701911 28897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:36.701982 28897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:36.702062 28897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:36.702112 28897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:36.713547 28897 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:41775
I20260810 22:39:36.714275 29036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:41775 every 8 connection(s)
I20260810 22:39:36.714406 28897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:36.714915 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 28897
I20260810 22:39:36.715075 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:36.715400 20265 external_mini_cluster.cc:1428] Setting key 0736371192d6aa3283d83da5dc76be91
I20260810 22:39:36.717157 28921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.703231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:32773 (local address 127.19.202.67:60035)
0810 22:39:36.703710 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.703749 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.704641 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.705234 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.705245 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.705643 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:36.706215 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.706229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.707355 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.707358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.708249 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.708258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.709023 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.709047 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.709247 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.709838 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:36.709863 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:36.712218 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:36.714187 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.714193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.714207 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.714582 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.714949 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.714952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.714954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.715106 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.715533 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:36.715539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:36.715784 (+ 245us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.715968 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.716282 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":140,"threads_started":1}
I20260810 22:39:36.717200 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.702737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:60035 (local address 127.19.202.126:32773)
0810 22:39:36.702863 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.702866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.703851 (+ 985us) server_negotiation.cc:407] Connection header received
0810 22:39:36.704894 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.704898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.704968 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.705068 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.706378 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.707242 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.708354 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.708765 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.708950 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.712387 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.712396 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.712398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.712426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.714061 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.714691 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.714695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.714697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.714826 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.715226 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.715228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.715229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.715402 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.715534 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.716925 (+ 1391us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.717059 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:39:36.718292 29037 heartbeater.cc:347] Connected to a master server at 127.19.202.126:32773
I20260810 22:39:36.718447 29037 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:36.718919 29037 heartbeater.cc:511] Master 127.19.202.126:32773 requested a full tablet report, sending...
I20260810 22:39:36.719941 28526 ts_manager.cc:194] Registered new tserver with Master: ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775)
I20260810 22:39:36.720672 28526 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:60035
I20260810 22:39:36.726576 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.720371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:58085 (local address 127.19.202.126:32773)
0810 22:39:36.720484 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.720488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.720667 (+ 179us) server_negotiation.cc:407] Connection header received
0810 22:39:36.720793 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.720796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.720905 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.721004 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.721380 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.721862 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.722633 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.722835 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.722941 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.723849 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.723852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.723855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.723876 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.725058 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.725474 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.725478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.725480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.725565 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.725827 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.725830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.725832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.725991 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.726091 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.726334 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.726435 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:36.726588 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.720285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:32773 (local address 127.19.202.67:58085)
0810 22:39:36.720573 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.720588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.720691 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.721030 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.721032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.721179 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:36.721267 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.721271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.721993 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.721998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.722522 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.722529 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.722939 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.722942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.723035 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.723442 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:36.723452 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:36.723719 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:36.725162 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.725167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.725169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.725361 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.725653 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:36.725656 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:36.725658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:36.725737 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:36.726084 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:36.726087 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:36.726178 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.726281 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.726409 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":107,"threads_started":1}
I20260810 22:39:36.729595 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:39:36.753783 28580 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:36.748045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.19.202.126:32773)
0810 22:39:36.748184 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.748187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.748200 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:36.748238 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.748240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.748276 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.748347 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.749057 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.749555 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.750338 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.750542 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.750614 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.753611 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52782: BlockingRecv error: recv got EOF from 127.0.0.1:52782 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 22:39:36 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:36.762859 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.755117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.19.202.126:32773)
0810 22:39:36.755249 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.755254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.755389 (+ 135us) server_negotiation.cc:407] Connection header received
0810 22:39:36.755537 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.755540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.755596 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.755668 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:36.756230 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.756753 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.757660 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.757897 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.758001 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.760218 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:36.760222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:36.760224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:36.760248 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:36.761552 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.761922 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.761924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.761925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.761997 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:36.762222 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:36.762224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:36.762225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:36.762368 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:36.762480 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.762678 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.762759 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:39:36.765588 28526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:36.768630 28526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:36.781826 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.776766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41341 (local address 127.0.0.1:59538)
0810 22:39:36.777184 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.777241 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.777361 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.778123 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.778130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.778153 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:36.778291 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.778301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.779369 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.779375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.780193 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.780202 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.781360 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.781364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.781472 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.781510 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.781642 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.781729 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":69,"threads_started":1}
I20260810 22:39:36.782217 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.776993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.19.202.66:41341)
0810 22:39:36.777402 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.777415 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.777441 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:39:36.777527 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.777534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.777751 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.777951 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:36.778432 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.779262 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.780319 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.781179 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.781270 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.781448 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.782023 (+ 575us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.782116 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":111,"threads_started":1}
I20260810 22:39:36.782877 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.777306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:46003 (local address 127.0.0.1:57548)
0810 22:39:36.777717 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.777732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.777827 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.778634 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.778637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.778653 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:36.778751 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.778757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.780203 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.780206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.781500 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.781507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.782522 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.782526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.782611 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.782633 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.782699 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.782760 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":147,"thread_start_us":115,"threads_started":1}
I20260810 22:39:36.782985 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.777768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.19.202.65:46003)
0810 22:39:36.778188 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.778202 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.778222 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:39:36.778298 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.778303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.778489 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.778892 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:36.779018 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.780085 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.781615 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.782338 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.782501 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.782676 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.782727 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.782819 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":116,"threads_started":1}
I20260810 22:39:36.785105 28819 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0c69b593404bd5b0b7521ab7dbef3b (DEFAULT_TABLE table=test-table [id=30a5be24c69b42218c3762271e353e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:36.785492 28671 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0c69b593404bd5b0b7521ab7dbef3b (DEFAULT_TABLE table=test-table [id=30a5be24c69b42218c3762271e353e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:36.786123 28819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0c69b593404bd5b0b7521ab7dbef3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:36.786396 28671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0c69b593404bd5b0b7521ab7dbef3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:36.791792 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.777554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:41775 (local address 127.0.0.1:47664)
0810 22:39:36.778009 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:36.778020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:36.778095 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:36.778865 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:36.778879 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:36.778896 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:36.778965 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.778969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.780334 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.780337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.781348 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:36.781358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.782622 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.782625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:36.782716 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.782741 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:39:36.782790 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 22:39:36.782854 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":26,"thread_start_us":98,"threads_started":1}
I20260810 22:39:36.792520 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:36.777986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.19.202.67:41775)
0810 22:39:36.778457 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:36.778468 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:36.778490 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:39:36.778560 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:36.778567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:36.778726 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:36.779110 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:36.779235 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.780227 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.781577 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:36.782494 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:36.792050 (+ 9556us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:36.792208 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:36.792261 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 22:39:36.792367 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":108,"threads_started":1}
I20260810 22:39:36.792972 29067 tablet_bootstrap.cc:492] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Bootstrap starting.
I20260810 22:39:36.793794 29066 tablet_bootstrap.cc:492] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Bootstrap starting.
I20260810 22:39:36.795387 29067 tablet_bootstrap.cc:654] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:36.795282 28963 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0c69b593404bd5b0b7521ab7dbef3b (DEFAULT_TABLE table=test-table [id=30a5be24c69b42218c3762271e353e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:36.796034 29066 tablet_bootstrap.cc:654] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:36.796314 28963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0c69b593404bd5b0b7521ab7dbef3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:36.796303 29067 log.cc:824] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:36.796939 29066 log.cc:824] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:36.798439 29067 tablet_bootstrap.cc:492] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: No bootstrap required, opened a new log
I20260810 22:39:36.798635 29067 ts_tablet_manager.cc:1397] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:39:36.798852 29066 tablet_bootstrap.cc:492] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: No bootstrap required, opened a new log
I20260810 22:39:36.799036 29066 ts_tablet_manager.cc:1397] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 22:39:36.802562 29067 raft_consensus.cc:348] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:36.802768 29070 tablet_bootstrap.cc:492] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Bootstrap starting.
I20260810 22:39:36.802863 29067 raft_consensus.cc:374] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:36.802738 29066 raft_consensus.cc:348] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:36.802935 29067 raft_consensus.cc:729] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ad79205310480095187ae07559510c, State: Initialized, Role: FOLLOWER
I20260810 22:39:36.802965 29066 raft_consensus.cc:374] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:36.803009 29066 raft_consensus.cc:729] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2757e381fe984fcdb38b34b7c2e7c685, State: Initialized, Role: FOLLOWER
I20260810 22:39:36.803417 29067 consensus_queue.cc:260] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:36.803417 29066 consensus_queue.cc:260] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:36.804286 28888 heartbeater.cc:503] Master 127.19.202.126:32773 was elected leader, sending a full tablet report...
I20260810 22:39:36.804426 29067 ts_tablet_manager.cc:1428] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 22:39:36.804419 29066 ts_tablet_manager.cc:1428] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:39:36.804768 28741 heartbeater.cc:503] Master 127.19.202.126:32773 was elected leader, sending a full tablet report...
I20260810 22:39:36.805081 29070 tablet_bootstrap.cc:654] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:36.805985 29070 log.cc:824] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:36.808032 29070 tablet_bootstrap.cc:492] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: No bootstrap required, opened a new log
I20260810 22:39:36.808225 29070 ts_tablet_manager.cc:1397] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:39:36.811316 29070 raft_consensus.cc:348] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:36.811533 29070 raft_consensus.cc:374] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:36.811566 29070 raft_consensus.cc:729] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Initialized, Role: FOLLOWER
I20260810 22:39:36.812019 29070 consensus_queue.cc:260] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:36.812654 29037 heartbeater.cc:503] Master 127.19.202.126:32773 was elected leader, sending a full tablet report...
I20260810 22:39:36.812875 29070 ts_tablet_manager.cc:1428] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260810 22:39:36.966560 29038 tablet.cc:2369] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:39:37.008985 28889 tablet.cc:2369] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:39:37.041754 29074 raft_consensus.cc:482] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260810 22:39:37.041935 28742 tablet.cc:2369] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:39:37.041971 29074 raft_consensus.cc:504] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:37.043149 29074 leader_election.cc:288] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ad79205310480095187ae07559510c (127.19.202.66:41341), 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003)
I20260810 22:39:37.047009 28921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.043355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41341 (local address 127.19.202.67:59999)
0810 22:39:37.043544 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.043558 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.043654 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.043906 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.043908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.043920 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:37.044063 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.044069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.044910 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.044912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.046006 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.046016 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.046693 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.046695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.046780 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.046801 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.046857 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.046911 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260810 22:39:37.047097 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.043444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:59999 (local address 127.19.202.66:41341)
0810 22:39:37.043548 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.043555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.043675 (+ 120us) server_negotiation.cc:407] Connection header received
0810 22:39:37.043738 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.043740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.043806 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.043885 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:37.044178 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.044804 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.046132 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.046586 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.046702 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.046751 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.046895 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.046968 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:39:37.047636 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.043511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:46241 (local address 127.19.202.65:46003)
0810 22:39:37.043630 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.043635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.043794 (+ 159us) server_negotiation.cc:407] Connection header received
0810 22:39:37.043851 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.043855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.043901 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.043969 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:37.044622 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.045224 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.046459 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.047009 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.047122 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.047176 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.047478 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.047544 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:37.047658 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.043403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:46003 (local address 127.19.202.67:46241)
0810 22:39:37.043701 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.043714 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.043794 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.043992 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.043995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.044006 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:37.044498 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.044506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.045333 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.045336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.046318 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.046325 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.047267 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.047270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.047353 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.047373 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.047456 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.047521 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":4,"thread_start_us":119,"threads_started":1}
I20260810 22:39:37.047622 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0c69b593404bd5b0b7521ab7dbef3b" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c" is_pre_election: true
I20260810 22:39:37.047941 28839 raft_consensus.cc:2454] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 0.
I20260810 22:39:37.048269 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0c69b593404bd5b0b7521ab7dbef3b" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685" is_pre_election: true
I20260810 22:39:37.048493 28914 leader_election.cc:302] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ad79205310480095187ae07559510c, ed981bc3e6284f2b9605fb26fbef0a03; no voters:
I20260810 22:39:37.048539 28693 raft_consensus.cc:2454] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 0.
I20260810 22:39:37.048755 29074 raft_consensus.cc:2781] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:37.048857 29074 raft_consensus.cc:482] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:37.048908 29074 raft_consensus.cc:3037] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.049952 29074 raft_consensus.cc:504] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:37.050268 29074 leader_election.cc:288] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 election: Requested vote from peers 30ad79205310480095187ae07559510c (127.19.202.66:41341), 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003)
I20260810 22:39:37.050558 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0c69b593404bd5b0b7521ab7dbef3b" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c"
I20260810 22:39:37.050705 28839 raft_consensus.cc:3037] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.050735 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0c69b593404bd5b0b7521ab7dbef3b" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685"
I20260810 22:39:37.050879 28693 raft_consensus.cc:3037] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.051687 28839 raft_consensus.cc:2454] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 1.
I20260810 22:39:37.051892 28693 raft_consensus.cc:2454] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 1.
I20260810 22:39:37.051970 28914 leader_election.cc:302] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ad79205310480095187ae07559510c, ed981bc3e6284f2b9605fb26fbef0a03; no voters:
I20260810 22:39:37.052152 29074 raft_consensus.cc:2781] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:37.052434 29074 raft_consensus.cc:686] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 LEADER]: Becoming Leader. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Running, Role: LEADER
I20260810 22:39:37.052747 29074 consensus_queue.cc:237] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } }
I20260810 22:39:37.055900 28526 catalog_manager.cc:5495] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:37.072532 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.069465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47672 (local address 127.19.202.67:41775)
0810 22:39:37.069573 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.069576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.069588 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:39:37.069684 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.069686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.069728 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.069808 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:37.070005 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.070486 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.071263 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.071469 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.071569 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.071812 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.072180 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.072273 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.072354 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.072441 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:39:37.078120 28839 raft_consensus.cc:1261] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Refusing update from remote peer ed981bc3e6284f2b9605fb26fbef0a03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:37.078120 28693 raft_consensus.cc:1261] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Refusing update from remote peer ed981bc3e6284f2b9605fb26fbef0a03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:37.078766 29076 consensus_queue.cc:1035] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:37.078955 29074 consensus_queue.cc:1035] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:37.086488 29080 mvcc.cc:205] Tried to move back new op lower bound from 7317100859704934400 to 7317100859610558464. Current Snapshot: MvccSnapshot[applied={T|T < 7317100859704934400}]
I20260810 22:39:37.096176 28527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:37.096346 28527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:37.098753 28527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Sending DeleteTablet for 3 replicas of tablet 2d0c69b593404bd5b0b7521ab7dbef3b
I20260810 22:39:37.099555 28963 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0c69b593404bd5b0b7521ab7dbef3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:47664
I20260810 22:39:37.099555 28671 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0c69b593404bd5b0b7521ab7dbef3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260810 22:39:37.099671 28819 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0c69b593404bd5b0b7521ab7dbef3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:59538
I20260810 22:39:37.100184 29091 tablet_replica.cc:330] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: stopping tablet replica
I20260810 22:39:37.100435 29092 tablet_replica.cc:330] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: stopping tablet replica
I20260810 22:39:37.100530 29091 raft_consensus.cc:2229] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:37.100708 29092 raft_consensus.cc:2229] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:37.100847 29091 raft_consensus.cc:2258] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:37.100769 28527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 22:39:37.100963 29092 raft_consensus.cc:2258] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 22:39:37.101197 28527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:37.101308 29089 tablet_replica.cc:330] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: stopping tablet replica
I20260810 22:39:37.101583 29089 raft_consensus.cc:2229] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:37.101914 29089 raft_consensus.cc:2258] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:37.102286 29091 ts_tablet_manager.cc:1905] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:37.103798 29092 ts_tablet_manager.cc:1905] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:37.105109 29091 ts_tablet_manager.cc:1918] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:37.105221 29091 log.cc:1196] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/2d0c69b593404bd5b0b7521ab7dbef3b
I20260810 22:39:37.105571 29091 ts_tablet_manager.cc:1939] T 2d0c69b593404bd5b0b7521ab7dbef3b P 2757e381fe984fcdb38b34b7c2e7c685: Deleting consensus metadata
I20260810 22:39:37.106667 28671 tablet_service.cc:1467] Processing CreateTablet for tablet 1471a65e348d41bdbe858943f6c50e1a (DEFAULT_TABLE table=test-table [id=00845316322849b8a21d5959f1277d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:37.106801 28819 tablet_service.cc:1467] Processing CreateTablet for tablet 1471a65e348d41bdbe858943f6c50e1a (DEFAULT_TABLE table=test-table [id=00845316322849b8a21d5959f1277d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:37.106946 28671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1471a65e348d41bdbe858943f6c50e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.107084 28819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1471a65e348d41bdbe858943f6c50e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.108737 29066 tablet_bootstrap.cc:492] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: Bootstrap starting.
I20260810 22:39:37.108925 28514 catalog_manager.cc:4841] TS 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003): tablet 2d0c69b593404bd5b0b7521ab7dbef3b (table test-table [id=30a5be24c69b42218c3762271e353e1b]) successfully deleted
I20260810 22:39:37.109620 28963 tablet_service.cc:1467] Processing CreateTablet for tablet 1471a65e348d41bdbe858943f6c50e1a (DEFAULT_TABLE table=test-table [id=00845316322849b8a21d5959f1277d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:37.109760 29066 tablet_bootstrap.cc:654] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.109870 28963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1471a65e348d41bdbe858943f6c50e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.110013 29089 ts_tablet_manager.cc:1905] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:37.111243 29067 tablet_bootstrap.cc:492] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: Bootstrap starting.
I20260810 22:39:37.111486 29066 tablet_bootstrap.cc:492] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: No bootstrap required, opened a new log
I20260810 22:39:37.111562 29066 ts_tablet_manager.cc:1397] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:37.112039 29066 raft_consensus.cc:348] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.112135 29066 raft_consensus.cc:374] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.112164 29066 raft_consensus.cc:729] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2757e381fe984fcdb38b34b7c2e7c685, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.112202 29067 tablet_bootstrap.cc:654] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.112279 29066 consensus_queue.cc:260] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.112524 29066 ts_tablet_manager.cc:1428] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 22:39:37.112900 29089 ts_tablet_manager.cc:1918] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:37.112995 29089 log.cc:1196] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/2d0c69b593404bd5b0b7521ab7dbef3b
I20260810 22:39:37.113315 29089 ts_tablet_manager.cc:1939] T 2d0c69b593404bd5b0b7521ab7dbef3b P ed981bc3e6284f2b9605fb26fbef0a03: Deleting consensus metadata
I20260810 22:39:37.114300 28512 catalog_manager.cc:4841] TS ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775): tablet 2d0c69b593404bd5b0b7521ab7dbef3b (table test-table [id=30a5be24c69b42218c3762271e353e1b]) successfully deleted
I20260810 22:39:37.114470 29070 tablet_bootstrap.cc:492] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: Bootstrap starting.
I20260810 22:39:37.115439 29070 tablet_bootstrap.cc:654] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.116755 29070 tablet_bootstrap.cc:492] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: No bootstrap required, opened a new log
I20260810 22:39:37.116882 29070 ts_tablet_manager.cc:1397] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:37.117307 29070 raft_consensus.cc:348] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.117434 29070 raft_consensus.cc:374] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.117470 29070 raft_consensus.cc:729] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.117581 29070 consensus_queue.cc:260] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.118213 29070 ts_tablet_manager.cc:1428] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:39:37.118948 29092 ts_tablet_manager.cc:1918] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:37.119046 29092 log.cc:1196] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/2d0c69b593404bd5b0b7521ab7dbef3b
I20260810 22:39:37.119885 29092 ts_tablet_manager.cc:1939] T 2d0c69b593404bd5b0b7521ab7dbef3b P 30ad79205310480095187ae07559510c: Deleting consensus metadata
I20260810 22:39:37.120975 28512 catalog_manager.cc:4841] TS 30ad79205310480095187ae07559510c (127.19.202.66:41341): tablet 2d0c69b593404bd5b0b7521ab7dbef3b (table test-table [id=30a5be24c69b42218c3762271e353e1b]) successfully deleted
I20260810 22:39:37.121600 29067 tablet_bootstrap.cc:492] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: No bootstrap required, opened a new log
I20260810 22:39:37.121706 29067 ts_tablet_manager.cc:1397] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260810 22:39:37.122133 29067 raft_consensus.cc:348] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.122243 29067 raft_consensus.cc:374] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.122272 29067 raft_consensus.cc:729] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ad79205310480095187ae07559510c, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.122442 29067 consensus_queue.cc:260] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.122690 29067 ts_tablet_manager.cc:1428] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 22:39:37.257005 29071 raft_consensus.cc:482] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:37.257164 29071 raft_consensus.cc:504] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.258291 29071 leader_election.cc:288] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ad79205310480095187ae07559510c (127.19.202.66:41341), ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775)
I20260810 22:39:37.262626 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.258647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:57491 (local address 127.19.202.66:41341)
0810 22:39:37.258797 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.258802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.258956 (+ 154us) server_negotiation.cc:407] Connection header received
0810 22:39:37.259163 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.259166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.259234 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.259314 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:37.259691 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.260237 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.261552 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.262085 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.262203 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.262264 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.262471 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.262529 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:37.262666 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.258584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41341 (local address 127.19.202.65:57491)
0810 22:39:37.258859 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.258871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.259003 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.259340 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.259343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.259362 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:37.259527 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.259535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.260316 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.260319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.261430 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.261440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.262213 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.262217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.262335 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.262362 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.262440 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.262509 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":121,"threads_started":1}
I20260810 22:39:37.263118 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1471a65e348d41bdbe858943f6c50e1a" candidate_uuid: "2757e381fe984fcdb38b34b7c2e7c685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c" is_pre_election: true
I20260810 22:39:37.263278 28839 raft_consensus.cc:2454] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2757e381fe984fcdb38b34b7c2e7c685 in term 0.
I20260810 22:39:37.263463 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.259011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:41775 (local address 127.19.202.65:51043)
0810 22:39:37.259570 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.259601 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.259739 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.260002 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.260005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.260021 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:37.260147 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.260153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.261126 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.261129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.262047 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.262058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.263021 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.263024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.263141 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.263166 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.263244 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.263319 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":108,"threads_started":1}
I20260810 22:39:37.263577 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.259115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:51043 (local address 127.19.202.67:41775)
0810 22:39:37.259249 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.259255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.259665 (+ 410us) server_negotiation.cc:407] Connection header received
0810 22:39:37.259840 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.259842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.259890 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.259980 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:37.260254 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.260999 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.262183 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.262770 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.262880 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.262954 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.263342 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.263446 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:37.263679 28621 leader_election.cc:302] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2757e381fe984fcdb38b34b7c2e7c685, 30ad79205310480095187ae07559510c; no voters:
I20260810 22:39:37.263906 29071 raft_consensus.cc:2781] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:37.263975 29071 raft_consensus.cc:482] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:37.264047 29071 raft_consensus.cc:3037] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.264295 28985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1471a65e348d41bdbe858943f6c50e1a" candidate_uuid: "2757e381fe984fcdb38b34b7c2e7c685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" is_pre_election: true
I20260810 22:39:37.264544 28985 raft_consensus.cc:2454] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2757e381fe984fcdb38b34b7c2e7c685 in term 0.
I20260810 22:39:37.265359 29071 raft_consensus.cc:504] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.265704 29071 leader_election.cc:288] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [CANDIDATE]: Term 1 election: Requested vote from peers 30ad79205310480095187ae07559510c (127.19.202.66:41341), ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775)
I20260810 22:39:37.265995 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1471a65e348d41bdbe858943f6c50e1a" candidate_uuid: "2757e381fe984fcdb38b34b7c2e7c685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c"
I20260810 22:39:37.266069 28985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1471a65e348d41bdbe858943f6c50e1a" candidate_uuid: "2757e381fe984fcdb38b34b7c2e7c685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed981bc3e6284f2b9605fb26fbef0a03"
I20260810 22:39:37.266141 28839 raft_consensus.cc:3037] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.266212 28985 raft_consensus.cc:3037] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.267007 28839 raft_consensus.cc:2454] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2757e381fe984fcdb38b34b7c2e7c685 in term 1.
I20260810 22:39:37.267144 28985 raft_consensus.cc:2454] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2757e381fe984fcdb38b34b7c2e7c685 in term 1.
I20260810 22:39:37.267278 28621 leader_election.cc:302] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2757e381fe984fcdb38b34b7c2e7c685, 30ad79205310480095187ae07559510c; no voters:
I20260810 22:39:37.267590 29071 raft_consensus.cc:2781] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:37.267788 29071 raft_consensus.cc:686] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 LEADER]: Becoming Leader. State: Replica: 2757e381fe984fcdb38b34b7c2e7c685, State: Running, Role: LEADER
I20260810 22:39:37.268047 29071 consensus_queue.cc:237] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.270155 28527 catalog_manager.cc:5495] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "2757e381fe984fcdb38b34b7c2e7c685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } health_report { overall_health: UNKNOWN } } }
Aug 10 22:39:37 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401575, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.126@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:37.334657 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.326340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:32773 (local address 127.0.0.1:52796)
0810 22:39:37.326701 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.326717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.326841 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.327085 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.327088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.327499 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:37.327580 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.327587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.328349 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.328354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.328960 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.328969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.329406 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.329409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.329507 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.330044 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:37.330055 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:37.331368 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:37.333046 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:37.333057 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:37.333063 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:37.333284 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:37.333601 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:37.333604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:37.333606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:37.333687 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:37.334090 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:37.334098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:37.334210 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.334332 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.334500 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":3,"thread_start_us":126,"threads_started":1}
I20260810 22:39:37.334885 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.326438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.19.202.126:32773)
0810 22:39:37.326718 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.326727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.326789 (+ 62us) server_negotiation.cc:407] Connection header received
0810 22:39:37.326940 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.326943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.326984 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.327070 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:37.327712 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.328243 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.329076 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.329305 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.329406 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.331489 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:37.331493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:37.331497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:37.331520 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:37.332912 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:37.333382 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:37.333387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:37.333389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:37.333493 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:37.333783 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:37.333786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:37.333788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:37.333971 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:37.334076 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.334378 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.334626 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":225,"thread_start_us":72,"threads_started":1}
I20260810 22:39:37.338624 28527 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:52796:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 22:39:37.339296 28527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:37.343921 28819 tablet_service.cc:1467] Processing CreateTablet for tablet a0fdaa6e7068453eae93452240f1a2ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=394b5e073f72496ea462ba7c9bec9c3f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:37.343926 28671 tablet_service.cc:1467] Processing CreateTablet for tablet a0fdaa6e7068453eae93452240f1a2ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=394b5e073f72496ea462ba7c9bec9c3f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:37.343987 28963 tablet_service.cc:1467] Processing CreateTablet for tablet a0fdaa6e7068453eae93452240f1a2ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=394b5e073f72496ea462ba7c9bec9c3f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:39:37.344318 28819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fdaa6e7068453eae93452240f1a2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.344324 28963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fdaa6e7068453eae93452240f1a2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.344329 28671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fdaa6e7068453eae93452240f1a2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.347143 29066 tablet_bootstrap.cc:492] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: Bootstrap starting.
I20260810 22:39:37.347179 29070 tablet_bootstrap.cc:492] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: Bootstrap starting.
I20260810 22:39:37.347936 29070 tablet_bootstrap.cc:654] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.347937 29066 tablet_bootstrap.cc:654] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.348300 29067 tablet_bootstrap.cc:492] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: Bootstrap starting.
I20260810 22:39:37.349161 29070 tablet_bootstrap.cc:492] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: No bootstrap required, opened a new log
I20260810 22:39:37.349237 29070 ts_tablet_manager.cc:1397] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 22:39:37.349334 29067 tablet_bootstrap.cc:654] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.349669 29070 raft_consensus.cc:348] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.349772 29070 raft_consensus.cc:374] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.349812 29070 raft_consensus.cc:729] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.349956 29070 consensus_queue.cc:260] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.350248 29070 ts_tablet_manager.cc:1428] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:39:37.350311 29066 tablet_bootstrap.cc:492] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: No bootstrap required, opened a new log
I20260810 22:39:37.350392 29066 ts_tablet_manager.cc:1397] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 22:39:37.350447 29076 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.350586 29067 tablet_bootstrap.cc:492] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: No bootstrap required, opened a new log
I20260810 22:39:37.350596 29076 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:37.350667 29067 ts_tablet_manager.cc:1397] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:37.350824 29066 raft_consensus.cc:348] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.350931 29066 raft_consensus.cc:374] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.350962 29066 raft_consensus.cc:729] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2757e381fe984fcdb38b34b7c2e7c685, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.351073 29066 consensus_queue.cc:260] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.351075 29067 raft_consensus.cc:348] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.351173 29067 raft_consensus.cc:374] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.351198 29067 raft_consensus.cc:729] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ad79205310480095187ae07559510c, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.351305 29067 consensus_queue.cc:260] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.351336 29066 ts_tablet_manager.cc:1428] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:37.351528 29067 ts_tablet_manager.cc:1428] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 22:39:37.351531 29071 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.351634 29071 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:37.352546 29072 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.352682 29072 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:37.669350 29071 consensus_queue.cc:1035] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:37.673493 29114 consensus_queue.cc:1035] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:37.676848 29072 raft_consensus.cc:482] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:37.676981 29072 raft_consensus.cc:504] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.678020 29072 leader_election.cc:288] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003), ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775)
I20260810 22:39:37.681592 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.678318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:39191 (local address 127.19.202.67:41775)
0810 22:39:37.678448 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.678452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.678602 (+ 150us) server_negotiation.cc:407] Connection header received
0810 22:39:37.678746 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.678749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.678806 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.678894 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:37.679171 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.679656 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.680641 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.681090 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.681206 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.681253 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.681426 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.681491 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:39:37.681613 29119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.678240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:41775 (local address 127.19.202.66:39191)
0810 22:39:37.678525 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.678537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.678647 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.678905 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.678908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.678923 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:37.679068 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.679074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.679760 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.679763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.680528 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.680536 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.681201 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.681204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.681295 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.681317 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.681411 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.681475 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":22,"thread_start_us":64,"threads_started":1}
I20260810 22:39:37.681866 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.678533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:59023 (local address 127.19.202.65:46003)
0810 22:39:37.678730 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.678734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.678845 (+ 111us) server_negotiation.cc:407] Connection header received
0810 22:39:37.678985 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.678988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.679036 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.679122 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:39:37.679368 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.679844 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.680813 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.681203 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.681330 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.681377 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.681536 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.681606 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260810 22:39:37.682013 29118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.678240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:46003 (local address 127.19.202.66:59023)
0810 22:39:37.678749 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.678761 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.678872 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.679136 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.679139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.679155 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:39:37.679255 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.679261 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.679963 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.679967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.680676 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.680685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.681327 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.681330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.681411 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.681432 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.681521 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.681585 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":94,"thread_start_us":67,"threads_started":1}
I20260810 22:39:37.681973 28985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdaa6e7068453eae93452240f1a2ec" candidate_uuid: "30ad79205310480095187ae07559510c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" is_pre_election: true
I20260810 22:39:37.682096 28985 raft_consensus.cc:2454] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ad79205310480095187ae07559510c in term 0.
I20260810 22:39:37.682446 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdaa6e7068453eae93452240f1a2ec" candidate_uuid: "30ad79205310480095187ae07559510c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685" is_pre_election: true
I20260810 22:39:37.682452 28767 leader_election.cc:302] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ad79205310480095187ae07559510c, ed981bc3e6284f2b9605fb26fbef0a03; no voters:
I20260810 22:39:37.682583 28693 raft_consensus.cc:2454] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ad79205310480095187ae07559510c in term 0.
I20260810 22:39:37.682677 29072 raft_consensus.cc:2781] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:37.682731 29072 raft_consensus.cc:482] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:37.682766 29072 raft_consensus.cc:3037] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.683687 29072 raft_consensus.cc:504] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.683976 29072 leader_election.cc:288] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [CANDIDATE]: Term 1 election: Requested vote from peers 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003), ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775)
I20260810 22:39:37.684259 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdaa6e7068453eae93452240f1a2ec" candidate_uuid: "30ad79205310480095187ae07559510c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685"
I20260810 22:39:37.684316 28985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdaa6e7068453eae93452240f1a2ec" candidate_uuid: "30ad79205310480095187ae07559510c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed981bc3e6284f2b9605fb26fbef0a03"
I20260810 22:39:37.684419 28693 raft_consensus.cc:3037] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.684432 28985 raft_consensus.cc:3037] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.685528 28985 raft_consensus.cc:2454] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ad79205310480095187ae07559510c in term 1.
I20260810 22:39:37.685528 28693 raft_consensus.cc:2454] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ad79205310480095187ae07559510c in term 1.
I20260810 22:39:37.685806 28769 leader_election.cc:302] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2757e381fe984fcdb38b34b7c2e7c685, 30ad79205310480095187ae07559510c; no voters:
I20260810 22:39:37.685989 29072 raft_consensus.cc:2781] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:37.686182 29072 raft_consensus.cc:686] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [term 1 LEADER]: Becoming Leader. State: Replica: 30ad79205310480095187ae07559510c, State: Running, Role: LEADER
I20260810 22:39:37.686407 29072 consensus_queue.cc:237] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.687583 29121 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: TxnStatusTablet state changed. Reason: New leader 30ad79205310480095187ae07559510c. Latest consensus state: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.687736 29121 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:37.687987 29124 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 22:39:37.688153 29124 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 22:39:37.688750 28526 catalog_manager.cc:5495] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ad79205310480095187ae07559510c (127.19.202.66). New cstate: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } health_report { overall_health: UNKNOWN } } }
I20260810 22:39:37.710177 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.706567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.19.202.66:41341)
0810 22:39:37.706877 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.706881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.706891 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:39:37.706950 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.706951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.707054 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.707172 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:37.707440 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.707900 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.708815 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.709029 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.709137 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.709440 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.709806 (+ 366us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.709946 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.710020 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.710064 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260810 22:39:37.710206 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.706482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41341 (local address 127.0.0.1:59554)
0810 22:39:37.706627 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.706647 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.706725 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.707199 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.707201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.707209 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:37.707316 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.707321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.708013 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.708017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.708658 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.708685 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.709138 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.709141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.709233 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.709279 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.709928 (+ 649us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.709944 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.710011 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.710093 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 22:39:37.712949 28693 raft_consensus.cc:1261] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Refusing update from remote peer 30ad79205310480095187ae07559510c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:37.713003 28985 raft_consensus.cc:1261] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Refusing update from remote peer 30ad79205310480095187ae07559510c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:37.713478 29072 consensus_queue.cc:1035] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:37.713673 29121 consensus_queue.cc:1035] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:37.715325 29114 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: TxnStatusTablet state changed. Reason: New leader 30ad79205310480095187ae07559510c. Latest consensus state: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.715449 29114 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:37.716457 29072 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.716559 29072 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:37.716696 29115 mvcc.cc:205] Tried to move back new op lower bound from 7317100862307328000 to 7317100862205906944. Current Snapshot: MvccSnapshot[applied={T|T < 7317100862307328000}]
I20260810 22:39:37.717406 29114 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.717507 29114 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P 2757e381fe984fcdb38b34b7c2e7c685: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:37.719048 29076 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: TxnStatusTablet state changed. Reason: New leader 30ad79205310480095187ae07559510c. Latest consensus state: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.719161 29076 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:37.726521 29121 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.726634 29121 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P 30ad79205310480095187ae07559510c: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:39:37.728070 29117 mvcc.cc:205] Tried to move back new op lower bound from 7317100862307328000 to 7317100862205906944. Current Snapshot: MvccSnapshot[applied={T|T < 7317100862307328000}]
I20260810 22:39:37.728243 29076 tablet_replica.cc:439] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30ad79205310480095187ae07559510c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } } }
I20260810 22:39:37.728343 29076 tablet_replica.cc:442] T a0fdaa6e7068453eae93452240f1a2ec P ed981bc3e6284f2b9605fb26fbef0a03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:39:37.731498 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.725443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.19.202.65:46003)
0810 22:39:37.725551 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.725555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.726162 (+ 607us) server_negotiation.cc:407] Connection header received
0810 22:39:37.726409 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.726413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.726480 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.726558 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:37.726973 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.727657 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.729924 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.730199 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.730270 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.730763 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.731122 (+ 359us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.731249 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.731304 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.731353 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:39:37.742139 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.738828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:58229 (local address 127.19.202.66:41341)
0810 22:39:37.738943 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.738947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.739091 (+ 144us) server_negotiation.cc:407] Connection header received
0810 22:39:37.739165 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.739168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.739214 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.739289 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:37.739522 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.740229 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.741143 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.741458 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.741552 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.741779 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.741850 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.741914 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.741983 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.742028 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:39:37.742152 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.738704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41341 (local address 127.19.202.65:58229)
0810 22:39:37.739003 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.739016 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.739097 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.739306 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.739308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.739313 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:37.739421 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.739424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.740329 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.740331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.741014 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.741023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.741559 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.741561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.741624 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.741664 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.741925 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.741930 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.741974 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.742031 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":117,"threads_started":1}
I20260810 22:39:37.766832 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.762970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:48551 (local address 127.19.202.65:46003)
0810 22:39:37.763095 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.763100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.763327 (+ 227us) server_negotiation.cc:407] Connection header received
0810 22:39:37.763442 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.763445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.763489 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.763567 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:37.763999 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.764827 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.765708 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.766033 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.766125 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.766379 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.766466 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.766538 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.766657 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.766705 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:37.766885 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.762879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:46003 (local address 127.19.202.66:48551)
0810 22:39:37.763233 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:37.763250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:37.763360 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:37.763592 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:37.763595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:37.763603 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:37.763874 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.763884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.764923 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.764928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.765584 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:37.765593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.766153 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.766156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:37.766223 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.766264 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.766567 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.766574 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 22:39:37.766641 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 22:39:37.766724 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":129,"threads_started":1}
I20260810 22:39:37.792623 28526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:37.792825 28526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:37.797482 28526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Sending DeleteTablet for 3 replicas of tablet 1471a65e348d41bdbe858943f6c50e1a
I20260810 22:39:37.798141 28671 tablet_service.cc:1514] Processing DeleteTablet for tablet 1471a65e348d41bdbe858943f6c50e1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260810 22:39:37.798971 28963 tablet_service.cc:1514] Processing DeleteTablet for tablet 1471a65e348d41bdbe858943f6c50e1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:47664
I20260810 22:39:37.799271 28819 tablet_service.cc:1514] Processing DeleteTablet for tablet 1471a65e348d41bdbe858943f6c50e1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:59538
I20260810 22:39:37.802735 29145 tablet_replica.cc:330] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: stopping tablet replica
I20260810 22:39:37.802901 29145 raft_consensus.cc:2229] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:37.803023 29145 raft_consensus.cc:2258] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:37.803844 29145 ts_tablet_manager.cc:1905] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:37.806727 29143 tablet_replica.cc:330] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: stopping tablet replica
I20260810 22:39:37.806856 29143 raft_consensus.cc:2229] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:37.807178 29143 raft_consensus.cc:2258] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:37.807413 29144 tablet_replica.cc:330] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: stopping tablet replica
I20260810 22:39:37.807551 29144 raft_consensus.cc:2229] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:37.807698 29144 raft_consensus.cc:2258] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:37.807868 29143 ts_tablet_manager.cc:1905] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:37.810134 29144 ts_tablet_manager.cc:1905] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:37.810853 29143 ts_tablet_manager.cc:1918] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:37.810931 29143 log.cc:1196] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/1471a65e348d41bdbe858943f6c50e1a
I20260810 22:39:37.811204 29143 ts_tablet_manager.cc:1939] T 1471a65e348d41bdbe858943f6c50e1a P 2757e381fe984fcdb38b34b7c2e7c685: Deleting consensus metadata
I20260810 22:39:37.811858 28514 catalog_manager.cc:4841] TS 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003): tablet 1471a65e348d41bdbe858943f6c50e1a (table test-table [id=00845316322849b8a21d5959f1277d63]) successfully deleted
I20260810 22:39:37.812640 29145 ts_tablet_manager.cc:1918] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:37.812721 29145 log.cc:1196] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/1471a65e348d41bdbe858943f6c50e1a
I20260810 22:39:37.813024 29145 ts_tablet_manager.cc:1939] T 1471a65e348d41bdbe858943f6c50e1a P 30ad79205310480095187ae07559510c: Deleting consensus metadata
I20260810 22:39:37.813273 29144 ts_tablet_manager.cc:1918] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:37.813397 29144 log.cc:1196] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/1471a65e348d41bdbe858943f6c50e1a
I20260810 22:39:37.813670 29144 ts_tablet_manager.cc:1939] T 1471a65e348d41bdbe858943f6c50e1a P ed981bc3e6284f2b9605fb26fbef0a03: Deleting consensus metadata
I20260810 22:39:37.813701 28512 catalog_manager.cc:4841] TS 30ad79205310480095187ae07559510c (127.19.202.66:41341): tablet 1471a65e348d41bdbe858943f6c50e1a (table test-table [id=00845316322849b8a21d5959f1277d63]) successfully deleted
I20260810 22:39:37.814374 28512 catalog_manager.cc:4841] TS ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775): tablet 1471a65e348d41bdbe858943f6c50e1a (table test-table [id=00845316322849b8a21d5959f1277d63]) successfully deleted
Aug 10 22:39:37 dist-test-slave-1x4g krb5kdc[28478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:39:37 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:37 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:39:37.841004 29108 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:37.829526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.19.202.126:32773)
0810 22:39:37.829662 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.829668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.833116 (+ 3448us) server_negotiation.cc:407] Connection header received
0810 22:39:37.833326 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.833330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.833382 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.833448 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:37.834462 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.835345 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.836695 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.837084 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.837205 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.840880 (+ 3675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52802: BlockingRecv error: recv got EOF from 127.0.0.1:52802 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 22:39:37 dist-test-slave-1x4g krb5kdc[28478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM
I20260810 22:39:37.852793 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.842943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.19.202.126:32773)
0810 22:39:37.843047 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.843050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.843267 (+ 217us) server_negotiation.cc:407] Connection header received
0810 22:39:37.843347 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.843349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.843391 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.843476 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:37.844142 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.844961 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.846103 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.846473 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.846581 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.849325 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:37.849328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:37.849331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:37.849354 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:37.850956 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:37.851597 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:37.851601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:37.851602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:37.851680 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:37.851927 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:37.851930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:37.851931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:37.852127 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:37.852216 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.852537 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.852653 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:39:37.855811 28526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:39:37.856267 28526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:37.861779 28671 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4adee2d22349b9b87026fad650602e (DEFAULT_TABLE table=test-table [id=664c09c553884203824435b09d52861b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:37.861963 28819 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4adee2d22349b9b87026fad650602e (DEFAULT_TABLE table=test-table [id=664c09c553884203824435b09d52861b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:37.862120 28671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4adee2d22349b9b87026fad650602e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.862236 28819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4adee2d22349b9b87026fad650602e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.864995 28963 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4adee2d22349b9b87026fad650602e (DEFAULT_TABLE table=test-table [id=664c09c553884203824435b09d52861b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:37.865264 28963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4adee2d22349b9b87026fad650602e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:37.865564 29160 tablet_bootstrap.cc:492] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: Bootstrap starting.
I20260810 22:39:37.866447 29160 tablet_bootstrap.cc:654] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.867551 29160 tablet_bootstrap.cc:492] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: No bootstrap required, opened a new log
I20260810 22:39:37.867621 29160 ts_tablet_manager.cc:1397] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:37.868027 29160 raft_consensus.cc:348] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.868139 29160 raft_consensus.cc:374] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.868163 29160 raft_consensus.cc:729] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2757e381fe984fcdb38b34b7c2e7c685, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.868261 29160 consensus_queue.cc:260] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.868482 29160 ts_tablet_manager.cc:1428] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:37.869431 29164 tablet_bootstrap.cc:492] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: Bootstrap starting.
I20260810 22:39:37.870375 29164 tablet_bootstrap.cc:654] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.871387 29159 tablet_bootstrap.cc:492] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: Bootstrap starting.
I20260810 22:39:37.872366 29159 tablet_bootstrap.cc:654] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:37.873183 29164 tablet_bootstrap.cc:492] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: No bootstrap required, opened a new log
I20260810 22:39:37.873270 29164 ts_tablet_manager.cc:1397] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 22:39:37.873651 29164 raft_consensus.cc:348] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.874277 29159 tablet_bootstrap.cc:492] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: No bootstrap required, opened a new log
I20260810 22:39:37.874375 29159 ts_tablet_manager.cc:1397] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 22:39:37.874781 29159 raft_consensus.cc:348] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.874898 29159 raft_consensus.cc:374] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.874930 29159 raft_consensus.cc:729] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ad79205310480095187ae07559510c, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.873759 29164 raft_consensus.cc:374] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:37.875087 29164 raft_consensus.cc:729] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Initialized, Role: FOLLOWER
I20260810 22:39:37.875221 29164 consensus_queue.cc:260] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.875535 29164 ts_tablet_manager.cc:1428] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:37.875910 29159 consensus_queue.cc:260] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.878039 29159 ts_tablet_manager.cc:1428] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260810 22:39:37.965749 29076 raft_consensus.cc:482] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:37.965982 29076 raft_consensus.cc:504] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.966356 29076 leader_election.cc:288] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003), 30ad79205310480095187ae07559510c (127.19.202.66:41341)
I20260810 22:39:37.966751 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4adee2d22349b9b87026fad650602e" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685" is_pre_election: true
I20260810 22:39:37.966809 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4adee2d22349b9b87026fad650602e" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c" is_pre_election: true
I20260810 22:39:37.966908 28693 raft_consensus.cc:2454] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 0.
I20260810 22:39:37.967002 28839 raft_consensus.cc:2454] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 0.
I20260810 22:39:37.967273 28916 leader_election.cc:302] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2757e381fe984fcdb38b34b7c2e7c685, ed981bc3e6284f2b9605fb26fbef0a03; no voters:
I20260810 22:39:37.967450 29076 raft_consensus.cc:2781] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:37.967514 29076 raft_consensus.cc:482] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:37.967541 29076 raft_consensus.cc:3037] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.968739 29076 raft_consensus.cc:504] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.969058 29076 leader_election.cc:288] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 election: Requested vote from peers 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003), 30ad79205310480095187ae07559510c (127.19.202.66:41341)
I20260810 22:39:37.969362 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4adee2d22349b9b87026fad650602e" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685"
I20260810 22:39:37.969383 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4adee2d22349b9b87026fad650602e" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c"
I20260810 22:39:37.969492 28693 raft_consensus.cc:3037] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.969494 28839 raft_consensus.cc:3037] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:37.970430 28693 raft_consensus.cc:2454] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 1.
I20260810 22:39:37.970573 28839 raft_consensus.cc:2454] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 1.
I20260810 22:39:37.970698 28916 leader_election.cc:302] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2757e381fe984fcdb38b34b7c2e7c685, ed981bc3e6284f2b9605fb26fbef0a03; no voters:
I20260810 22:39:37.970908 29076 raft_consensus.cc:2781] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:37.971079 29076 raft_consensus.cc:686] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 LEADER]: Becoming Leader. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Running, Role: LEADER
I20260810 22:39:37.971241 29076 consensus_queue.cc:237] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:37.972944 28524 catalog_manager.cc:5495] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } health_report { overall_health: HEALTHY } } }
I20260810 22:39:37.994217 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:37.991102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47688 (local address 127.19.202.67:41775)
0810 22:39:37.991222 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:37.991229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:37.991264 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:39:37.991421 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:37.991424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:37.991478 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:37.991561 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:37.991795 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.992294 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.993234 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:37.993453 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:37.993551 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:37.993797 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:37.993866 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:37.993952 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:37.994049 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 22:39:37.994098 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:39:37.996142 28839 raft_consensus.cc:1261] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Refusing update from remote peer ed981bc3e6284f2b9605fb26fbef0a03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:37.996157 28693 raft_consensus.cc:1261] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Refusing update from remote peer ed981bc3e6284f2b9605fb26fbef0a03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:37.996605 29076 consensus_queue.cc:1035] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:37.996765 29167 consensus_queue.cc:1035] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:38.003425 28524 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:38.003554 28524 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:38.005262 28524 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Sending DeleteTablet for 3 replicas of tablet 8c4adee2d22349b9b87026fad650602e
I20260810 22:39:38.005775 28671 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4adee2d22349b9b87026fad650602e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260810 22:39:38.005868 28819 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4adee2d22349b9b87026fad650602e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:59538
I20260810 22:39:38.005968 29143 tablet_replica.cc:330] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: stopping tablet replica
I20260810 22:39:38.005980 28963 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4adee2d22349b9b87026fad650602e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:37 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:47664
I20260810 22:39:38.006024 29145 tablet_replica.cc:330] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: stopping tablet replica
I20260810 22:39:38.006076 29143 raft_consensus.cc:2229] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:38.006109 29145 raft_consensus.cc:2229] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:38.006131 29144 tablet_replica.cc:330] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: stopping tablet replica
I20260810 22:39:38.006181 29143 raft_consensus.cc:2258] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:38.006206 29145 raft_consensus.cc:2258] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:38.006225 29144 raft_consensus.cc:2229] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:38.006381 29144 raft_consensus.cc:2258] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:38.006904 29143 ts_tablet_manager.cc:1905] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:38.006933 28524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 22:39:38.007072 29144 ts_tablet_manager.cc:1905] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 22:39:38.007292 28524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:39:38.008378 29145 ts_tablet_manager.cc:1905] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:38.009655 29144 ts_tablet_manager.cc:1918] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:38.009738 29144 log.cc:1196] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/8c4adee2d22349b9b87026fad650602e
I20260810 22:39:38.009996 29144 ts_tablet_manager.cc:1939] T 8c4adee2d22349b9b87026fad650602e P ed981bc3e6284f2b9605fb26fbef0a03: Deleting consensus metadata
I20260810 22:39:38.010650 28512 catalog_manager.cc:4841] TS ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775): tablet 8c4adee2d22349b9b87026fad650602e (table test-table [id=664c09c553884203824435b09d52861b]) successfully deleted
I20260810 22:39:38.010700 29143 ts_tablet_manager.cc:1918] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:38.010774 29143 log.cc:1196] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/8c4adee2d22349b9b87026fad650602e
I20260810 22:39:38.010833 29145 ts_tablet_manager.cc:1918] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:39:38.010901 29145 log.cc:1196] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/8c4adee2d22349b9b87026fad650602e
I20260810 22:39:38.011036 29143 ts_tablet_manager.cc:1939] T 8c4adee2d22349b9b87026fad650602e P 2757e381fe984fcdb38b34b7c2e7c685: Deleting consensus metadata
I20260810 22:39:38.011174 29145 ts_tablet_manager.cc:1939] T 8c4adee2d22349b9b87026fad650602e P 30ad79205310480095187ae07559510c: Deleting consensus metadata
I20260810 22:39:38.011773 28514 catalog_manager.cc:4841] TS 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003): tablet 8c4adee2d22349b9b87026fad650602e (table test-table [id=664c09c553884203824435b09d52861b]) successfully deleted
I20260810 22:39:38.011869 28671 tablet_service.cc:1467] Processing CreateTablet for tablet f99b05a0033d4ab0b03c2d1aeecdfa52 (DEFAULT_TABLE table=test-table [id=24489b6af0a54079ad43edd3ec3ba698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:38.012096 28671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99b05a0033d4ab0b03c2d1aeecdfa52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:38.012352 28819 tablet_service.cc:1467] Processing CreateTablet for tablet f99b05a0033d4ab0b03c2d1aeecdfa52 (DEFAULT_TABLE table=test-table [id=24489b6af0a54079ad43edd3ec3ba698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:38.012559 28819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99b05a0033d4ab0b03c2d1aeecdfa52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:38.013211 28963 tablet_service.cc:1467] Processing CreateTablet for tablet f99b05a0033d4ab0b03c2d1aeecdfa52 (DEFAULT_TABLE table=test-table [id=24489b6af0a54079ad43edd3ec3ba698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:39:38.013556 28963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99b05a0033d4ab0b03c2d1aeecdfa52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:38.014261 29160 tablet_bootstrap.cc:492] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685: Bootstrap starting.
I20260810 22:39:38.014991 29160 tablet_bootstrap.cc:654] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:38.015563 29164 tablet_bootstrap.cc:492] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: Bootstrap starting.
I20260810 22:39:38.016004 29160 tablet_bootstrap.cc:492] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685: No bootstrap required, opened a new log
I20260810 22:39:38.016109 29160 ts_tablet_manager.cc:1397] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:38.016259 29164 tablet_bootstrap.cc:654] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:38.016520 29160 raft_consensus.cc:348] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.016678 29160 raft_consensus.cc:374] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:38.016727 29160 raft_consensus.cc:729] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2757e381fe984fcdb38b34b7c2e7c685, State: Initialized, Role: FOLLOWER
I20260810 22:39:38.016873 29160 consensus_queue.cc:260] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.017203 29160 ts_tablet_manager.cc:1428] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:38.018685 29159 tablet_bootstrap.cc:492] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: Bootstrap starting.
I20260810 22:39:38.018824 28512 catalog_manager.cc:4841] TS 30ad79205310480095187ae07559510c (127.19.202.66:41341): tablet 8c4adee2d22349b9b87026fad650602e (table test-table [id=664c09c553884203824435b09d52861b]) successfully deleted
I20260810 22:39:38.019488 29164 tablet_bootstrap.cc:492] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: No bootstrap required, opened a new log
I20260810 22:39:38.019490 29159 tablet_bootstrap.cc:654] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:38.019569 29164 ts_tablet_manager.cc:1397] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 22:39:38.019936 29164 raft_consensus.cc:348] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.020037 29164 raft_consensus.cc:374] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:38.020069 29164 raft_consensus.cc:729] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Initialized, Role: FOLLOWER
I20260810 22:39:38.020172 29164 consensus_queue.cc:260] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.020378 29164 ts_tablet_manager.cc:1428] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:39:38.027070 29159 tablet_bootstrap.cc:492] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: No bootstrap required, opened a new log
I20260810 22:39:38.027195 29159 ts_tablet_manager.cc:1397] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260810 22:39:38.027573 29159 raft_consensus.cc:348] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.027688 29159 raft_consensus.cc:374] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:38.027732 29159 raft_consensus.cc:729] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ad79205310480095187ae07559510c, State: Initialized, Role: FOLLOWER
I20260810 22:39:38.027864 29159 consensus_queue.cc:260] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.028163 29159 ts_tablet_manager.cc:1428] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 22:39:38.049437 29076 raft_consensus.cc:482] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:39:38.049537 29076 raft_consensus.cc:504] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.049887 29076 leader_election.cc:288] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003), 30ad79205310480095187ae07559510c (127.19.202.66:41341)
I20260810 22:39:38.050239 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99b05a0033d4ab0b03c2d1aeecdfa52" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c" is_pre_election: true
I20260810 22:39:38.050253 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99b05a0033d4ab0b03c2d1aeecdfa52" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685" is_pre_election: true
I20260810 22:39:38.050366 28839 raft_consensus.cc:2454] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 0.
I20260810 22:39:38.050405 28693 raft_consensus.cc:2454] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 0.
I20260810 22:39:38.050691 28914 leader_election.cc:302] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ad79205310480095187ae07559510c, ed981bc3e6284f2b9605fb26fbef0a03; no voters:
I20260810 22:39:38.050863 29076 raft_consensus.cc:2781] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:39:38.050922 29076 raft_consensus.cc:482] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:39:38.050948 29076 raft_consensus.cc:3037] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:38.051826 29076 raft_consensus.cc:504] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.052177 29076 leader_election.cc:288] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 election: Requested vote from peers 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003), 30ad79205310480095187ae07559510c (127.19.202.66:41341)
I20260810 22:39:38.052486 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99b05a0033d4ab0b03c2d1aeecdfa52" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2757e381fe984fcdb38b34b7c2e7c685"
I20260810 22:39:38.052495 28839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99b05a0033d4ab0b03c2d1aeecdfa52" candidate_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad79205310480095187ae07559510c"
I20260810 22:39:38.052596 28693 raft_consensus.cc:3037] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:38.052610 28839 raft_consensus.cc:3037] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:38.053453 28693 raft_consensus.cc:2454] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 1.
I20260810 22:39:38.053519 28839 raft_consensus.cc:2454] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed981bc3e6284f2b9605fb26fbef0a03 in term 1.
I20260810 22:39:38.053737 28916 leader_election.cc:302] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2757e381fe984fcdb38b34b7c2e7c685, ed981bc3e6284f2b9605fb26fbef0a03; no voters:
I20260810 22:39:38.053908 29076 raft_consensus.cc:2781] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:38.054042 29076 raft_consensus.cc:686] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 LEADER]: Becoming Leader. State: Replica: ed981bc3e6284f2b9605fb26fbef0a03, State: Running, Role: LEADER
I20260810 22:39:38.054186 29076 consensus_queue.cc:237] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } }
I20260810 22:39:38.055711 28526 catalog_manager.cc:5495] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed981bc3e6284f2b9605fb26fbef0a03" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 41775 } health_report { overall_health: HEALTHY } } }
I20260810 22:39:38.079325 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:38.076309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:37745 (local address 127.19.202.66:41341)
0810 22:39:38.076432 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:38.076435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:38.076632 (+ 197us) server_negotiation.cc:407] Connection header received
0810 22:39:38.076748 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:38.076750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:38.076847 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:38.076929 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:38.077151 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.077633 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.078476 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.078672 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.078752 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:38.078991 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:38.079051 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:38.079098 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:38.079186 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 22:39:38.079226 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:39:38.079361 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:38.076206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41341 (local address 127.19.202.67:37745)
0810 22:39:38.076527 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:38.076540 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:38.076665 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:38.076945 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:38.076947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:38.076955 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:38.077053 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:38.077059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.077726 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.077728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:38.078371 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:38.078381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.078774 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.078777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:38.078839 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:38.078884 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:38.079119 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:38.079124 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:39:38.079176 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 22:39:38.079243 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
I20260810 22:39:38.084740 28839 raft_consensus.cc:1261] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Refusing update from remote peer ed981bc3e6284f2b9605fb26fbef0a03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:38.084741 28693 raft_consensus.cc:1261] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Refusing update from remote peer ed981bc3e6284f2b9605fb26fbef0a03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:39:38.085150 29167 consensus_queue.cc:1035] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2757e381fe984fcdb38b34b7c2e7c685" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:38.085287 29076 consensus_queue.cc:1035] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ad79205310480095187ae07559510c" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:39:38.102066 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:38.099018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:44289 (local address 127.19.202.67:41775)
0810 22:39:38.099124 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:38.099128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:38.099181 (+ 53us) server_negotiation.cc:407] Connection header received
0810 22:39:38.099323 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:38.099326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:38.099389 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:38.099466 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:39:38.099656 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.100110 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.100864 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.101190 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.101306 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:38.101596 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:39:38.101681 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:39:38.101759 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:38.101856 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 22:39:38.101929 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:39:38.102090 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:38.098967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:41775 (local address 127.19.202.66:44289)
0810 22:39:38.099119 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:38.099134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:38.099221 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:38.099493 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:38.099495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:38.099502 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:39:38.099587 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:38.099592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.100193 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.100195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:38.100727 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:38.100755 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:38.101316 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:38.101319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:38.101378 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:38.101468 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:39:38.101774 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:39:38.101781 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 22:39:38.101852 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 22:39:38.101959 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 22:39:38.120882 28526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:38.121022 28526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:39:38.122973 28526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c72f50ae93a4020b8352fdabff2eb76: Sending DeleteTablet for 3 replicas of tablet f99b05a0033d4ab0b03c2d1aeecdfa52
I20260810 22:39:38.123536 28819 tablet_service.cc:1514] Processing DeleteTablet for tablet f99b05a0033d4ab0b03c2d1aeecdfa52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:38 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:59538
I20260810 22:39:38.123549 28963 tablet_service.cc:1514] Processing DeleteTablet for tablet f99b05a0033d4ab0b03c2d1aeecdfa52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:38 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:47664
I20260810 22:39:38.123541 28671 tablet_service.cc:1514] Processing DeleteTablet for tablet f99b05a0033d4ab0b03c2d1aeecdfa52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:39:38 UTC) from {username='oryx', principal='oryx/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260810 22:39:38.123723 29144 tablet_replica.cc:330] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: stopping tablet replica
I20260810 22:39:38.123852 29144 raft_consensus.cc:2229] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:39:38.123956 29145 tablet_replica.cc:330] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: stopping tablet replica
I20260810 22:39:38.124015 29144 raft_consensus.cc:2258] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:38.124082 29145 raft_consensus.cc:2229] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:38.124186 29145 raft_consensus.cc:2258] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:38.124183 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 28603
I20260810 22:39:38.124204 29143 tablet_replica.cc:330] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685: stopping tablet replica
I20260810 22:39:38.124341 29143 raft_consensus.cc:2229] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:39:38.124462 29143 raft_consensus.cc:2258] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 2757e381fe984fcdb38b34b7c2e7c685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:39:38.125838 29145 ts_tablet_manager.cc:1905] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:38.128686 29144 ts_tablet_manager.cc:1905] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:39:38.128804 29145 ts_tablet_manager.cc:1918] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:38.128875 29145 log.cc:1196] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/f99b05a0033d4ab0b03c2d1aeecdfa52
I20260810 22:39:38.129132 29145 ts_tablet_manager.cc:1939] T f99b05a0033d4ab0b03c2d1aeecdfa52 P 30ad79205310480095187ae07559510c: Deleting consensus metadata
I20260810 22:39:38.129802 28512 catalog_manager.cc:4841] TS 30ad79205310480095187ae07559510c (127.19.202.66:41341): tablet f99b05a0033d4ab0b03c2d1aeecdfa52 (table test-table [id=24489b6af0a54079ad43edd3ec3ba698]) successfully deleted
I20260810 22:39:38.130879 29144 ts_tablet_manager.cc:1918] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:39:38.130972 29144 log.cc:1196] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/f99b05a0033d4ab0b03c2d1aeecdfa52
I20260810 22:39:38.131212 29144 ts_tablet_manager.cc:1939] T f99b05a0033d4ab0b03c2d1aeecdfa52 P ed981bc3e6284f2b9605fb26fbef0a03: Deleting consensus metadata
I20260810 22:39:38.131841 28512 catalog_manager.cc:4841] TS ed981bc3e6284f2b9605fb26fbef0a03 (127.19.202.67:41775): tablet f99b05a0033d4ab0b03c2d1aeecdfa52 (table test-table [id=24489b6af0a54079ad43edd3ec3ba698]) successfully deleted
W20260810 22:39:38.135918 28514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:46003 (error 108)
I20260810 22:39:38.136822 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 28750
I20260810 22:39:38.137176 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:38.136689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:46003 (local address 127.0.0.1:57576)
0810 22:39:38.136975 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:38.137058 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:46003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":132,"threads_started":1}
W20260810 22:39:38.137921 28514 catalog_manager.cc:4584] TS 2757e381fe984fcdb38b34b7c2e7c685 (127.19.202.65:46003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f99b05a0033d4ab0b03c2d1aeecdfa52: Network error: Client connection negotiation failed: client connection to 127.19.202.65:46003: connect: Connection refused (error 111)
I20260810 22:39:38.145812 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 28897
I20260810 22:39:38.154098 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 28492
2026-08-10T22:39:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3471 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:38 dist-test-slave-1x4g krb5kdc[29188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:38 dist-test-slave-1x4g krb5kdc[29188](info): set up 2 sockets
Aug 10 22:39:38 dist-test-slave-1x4g krb5kdc[29188](info): commencing operation
krb5kdc: starting...
W20260810 22:39:39.226475 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:39 dist-test-slave-1x4g krb5kdc[29188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:39.347149 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39373
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:39373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:39.454659 29202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:39.454944 29202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:39.455026 29202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:39.459013 29202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:39.459089 29202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:39.459137 29202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:39.459208 29202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:39.459242 29202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:39.464020 29202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:39373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:39.465054 29202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:39.466045 29202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:39.474327 29210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:39.474351 29208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:39.474327 29207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:39.475193 29202 server_base.cc:1034] running on GCE node
I20260810 22:39:39.475796 29202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:39.476948 29202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:39.478169 29202 hybrid_clock.cc:648] HybridClock initialized: now 1786401579478127 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T22:39:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1326 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:39 dist-test-slave-1x4g krb5kdc[29215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:39 dist-test-slave-1x4g krb5kdc[29215](info): set up 2 sockets
Aug 10 22:39:39 dist-test-slave-1x4g krb5kdc[29215](info): commencing operation
krb5kdc: starting...
W20260810 22:39:40.556021 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:39:40 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:39:40.679800 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36639
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:36639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:40.786269 29229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:40.786550 29229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:40.786633 29229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:40.790985 29229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:40.791050 29229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:40.791083 29229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:40.791122 29229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:40.791154 29229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:40.797606 29229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:36639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:40.799057 29229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:40.800427 29229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:40.811034 29235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:40.811034 29234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:40.811219 29229 server_base.cc:1034] running on GCE node
W20260810 22:39:40.811523 29237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:40.812186 29229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:40.813730 29229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:40.814934 29229 hybrid_clock.cc:648] HybridClock initialized: now 1786401580814928 us; error 56 us; skew 500 ppm
Aug 10 22:39:40 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401580, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:40.818157 29229 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:40.819527 29229 webserver.cc:460] Webserver started at http://127.19.202.126:39853/ using document root <none> and password file <none>
I20260810 22:39:40.820220 29229 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:40.820338 29229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:40.820624 29229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:40.823460 29229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "21739e962b324429bbbaf6b628e40ed0"
format_stamp: "Formatted at 2026-08-10 22:39:40 on dist-test-slave-1x4g"
server_key: "e1de3f0004b6ff980bb4901fdd7616f4"
server_key_iv: "6f48d102ff8e58f7538f6ae5986ef328"
server_key_version: "encryptionkey@0"
I20260810 22:39:40.824172 29229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "21739e962b324429bbbaf6b628e40ed0"
format_stamp: "Formatted at 2026-08-10 22:39:40 on dist-test-slave-1x4g"
server_key: "e1de3f0004b6ff980bb4901fdd7616f4"
server_key_iv: "6f48d102ff8e58f7538f6ae5986ef328"
server_key_version: "encryptionkey@0"
I20260810 22:39:40.829346 29229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 22:39:40.832521 29244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:40.834230 29229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:40.834373 29229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "21739e962b324429bbbaf6b628e40ed0"
format_stamp: "Formatted at 2026-08-10 22:39:40 on dist-test-slave-1x4g"
server_key: "e1de3f0004b6ff980bb4901fdd7616f4"
server_key_iv: "6f48d102ff8e58f7538f6ae5986ef328"
server_key_version: "encryptionkey@0"
I20260810 22:39:40.834507 29229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:40.850142 29229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:40.853372 29229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:40.853528 29229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:40.860713 29229 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:36639
I20260810 22:39:40.860751 29300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:36639 every 8 connection(s)
I20260810 22:39:40.861690 29229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:40.864342 29301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:40.866070 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 29229
I20260810 22:39:40.866212 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:40.866528 20265 external_mini_cluster.cc:1428] Setting key cbf4152a2e9cd5b2219eba35f75c3cde
I20260810 22:39:40.870733 29301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Bootstrap starting.
I20260810 22:39:40.872987 29301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:40.873907 29301 log.cc:824] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Log is configured to *not* fsync() on all Append() calls
Aug 10 22:39:40 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:40.876010 29301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: No bootstrap required, opened a new log
I20260810 22:39:40.880767 29301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:40.881050 29301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:40.881090 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:40.867979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.19.202.126:36639)
0810 22:39:40.868408 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:40.868423 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:40.868457 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:40.869134 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:40.869152 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:40.869472 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:40.869793 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:40.870604 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:40.871595 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:40.872433 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:40.873005 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:40.873184 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:40.875931 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:40.875937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:40.875953 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:40.875982 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:40.877988 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:40.878400 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:40.878408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:40.878415 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:40.878596 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:40.878860 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:40.878863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:40.878865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:40.879339 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:40.879556 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:40.879750 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 22:39:40.880048 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
I20260810 22:39:40.881119 29301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21739e962b324429bbbaf6b628e40ed0, State: Initialized, Role: FOLLOWER
I20260810 22:39:40.881587 29301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:40.881743 29301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:40.881841 29301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:40.881940 29301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:40.883031 29301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:40.883450 29301 leader_election.cc:302] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21739e962b324429bbbaf6b628e40ed0; no voters:
I20260810 22:39:40.883769 29301 leader_election.cc:288] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:40.883873 29306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:40.884136 29306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 LEADER]: Becoming Leader. State: Replica: 21739e962b324429bbbaf6b628e40ed0, State: Running, Role: LEADER
I20260810 22:39:40.884415 29306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:40.885061 29301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:40.886128 29310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21739e962b324429bbbaf6b628e40ed0. Latest consensus state: current_term: 1 leader_uuid: "21739e962b324429bbbaf6b628e40ed0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } } }
I20260810 22:39:40.886258 29310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:40.886546 29308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21739e962b324429bbbaf6b628e40ed0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } } }
I20260810 22:39:40.886641 29308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:40.886729 29317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:40.889848 29317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:40.895551 29317 catalog_manager.cc:1347] Generated new cluster ID: 95015009dce04951af8835a3003a04ee
I20260810 22:39:40.895622 29317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:40.904047 29317 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:40.905035 29317 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:40.915226 29317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Generated new TSK 0
I20260810 22:39:40.915971 29317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:39:40.953328 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36639
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:41.066798 29331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:41.067090 29331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:41.067171 29331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:41.071094 29331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:41.071183 29331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:41.071277 29331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:41.076367 29331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:41.077458 29331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:41.078563 29331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:41.088305 29337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.088305 29339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.088315 29336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:41.088387 29331 server_base.cc:1034] running on GCE node
I20260810 22:39:41.089035 29331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:41.089668 29331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:41.090860 29331 hybrid_clock.cc:648] HybridClock initialized: now 1786401581090845 us; error 29 us; skew 500 ppm
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:41.098099 29331 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:39:41.099227 29331 webserver.cc:460] Webserver started at http://127.19.202.65:45243/ using document root <none> and password file <none>
I20260810 22:39:41.099829 29331 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:41.099929 29331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:41.100162 29331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:41.102283 29331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "bec224e543bf4697b8385a9eaf3a8997"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "2105cf02380f516320996a5c1ca9911e"
server_key_iv: "8418d8c918995586f2b64387017fda24"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.102855 29331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "bec224e543bf4697b8385a9eaf3a8997"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "2105cf02380f516320996a5c1ca9911e"
server_key_iv: "8418d8c918995586f2b64387017fda24"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.106940 29331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 22:39:41.109254 29346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.110525 29331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:41.110668 29331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "bec224e543bf4697b8385a9eaf3a8997"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "2105cf02380f516320996a5c1ca9911e"
server_key_iv: "8418d8c918995586f2b64387017fda24"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.110777 29331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:41.124934 29331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:41.128203 29331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:41.128414 29331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:41.129069 29331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:41.130008 29331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:41.130076 29331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.130143 29331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:41.130190 29331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.139623 29331 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:44239
I20260810 22:39:41.139700 29471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:44239 every 8 connection(s)
I20260810 22:39:41.140614 29331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:41.150254 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 29331
I20260810 22:39:41.150390 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:41.150707 20265 external_mini_cluster.cc:1428] Setting key 0b2fe52812257b490ab340763683bb34
I20260810 22:39:41.155599 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:41.142539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:45283 (local address 127.19.202.126:36639)
0810 22:39:41.142694 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:41.142698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:41.143517 (+ 819us) server_negotiation.cc:407] Connection header received
0810 22:39:41.145048 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:41.145052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:41.145120 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:41.145236 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:41.146431 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.146990 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.148155 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.148406 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.148530 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:41.151342 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:41.151346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:41.151349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:41.151389 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:41.153290 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:41.153814 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:41.153818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:41.153821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:41.153929 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:41.154271 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:41.154277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:41.154282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:41.154569 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:41.154700 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:41.155349 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 22:39:41.155471 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 22:39:41.156642 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:41.142917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:45283)
0810 22:39:41.143365 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:41.143409 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:41.144749 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:41.145417 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:41.145424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:41.145782 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:41.146266 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:41.146280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.147132 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.147142 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:41.148016 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:41.148026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.148571 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.148576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:41.148750 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:41.149372 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:41.149390 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:41.151133 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:41.153429 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:41.153434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:41.153445 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:41.153705 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:41.154034 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:41.154036 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:41.154037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:41.154173 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:41.154711 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:41.154719 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:41.155067 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 22:39:41.155294 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 22:39:41.155657 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":119,"threads_started":1}
I20260810 22:39:41.158025 29472 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36639
I20260810 22:39:41.158303 29472 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:41.158959 29472 heartbeater.cc:511] Master 127.19.202.126:36639 requested a full tablet report, sending...
I20260810 22:39:41.160637 29262 ts_manager.cc:194] Registered new tserver with Master: bec224e543bf4697b8385a9eaf3a8997 (127.19.202.65:44239)
I20260810 22:39:41.161800 29262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:45283
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:39:41.181334 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36639
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:41.292709 29478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:41.293004 29478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:41.293093 29478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:41.297020 29478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:41.297098 29478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:41.297187 29478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:41.302199 29478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:41.303193 29478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:41.304201 29478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:41.313458 29483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.313452 29486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:41.313720 29478 server_base.cc:1034] running on GCE node
W20260810 22:39:41.313452 29484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:41.314178 29478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:41.314843 29478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:41.316013 29478 hybrid_clock.cc:648] HybridClock initialized: now 1786401581315999 us; error 23 us; skew 500 ppm
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:41.318872 29478 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:39:41.319695 29478 webserver.cc:460] Webserver started at http://127.19.202.66:43181/ using document root <none> and password file <none>
I20260810 22:39:41.320227 29478 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:41.320338 29478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:41.320603 29478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:41.322624 29478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "bcf04ae33dbb4680bb38c11ee658e43d"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "100f484c33ba8c4ebd527fe2b3c2dc0d"
server_key_iv: "9bbd25ba7387dc4190d358efa250cc36"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.323196 29478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "bcf04ae33dbb4680bb38c11ee658e43d"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "100f484c33ba8c4ebd527fe2b3c2dc0d"
server_key_iv: "9bbd25ba7387dc4190d358efa250cc36"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.326820 29478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 22:39:41.329221 29494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.330421 29478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:41.330554 29478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "bcf04ae33dbb4680bb38c11ee658e43d"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "100f484c33ba8c4ebd527fe2b3c2dc0d"
server_key_iv: "9bbd25ba7387dc4190d358efa250cc36"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.330659 29478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:41.346683 29478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:41.349853 29478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:41.350054 29478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:41.350622 29478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:41.351428 29478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:41.351497 29478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.351567 29478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:41.351609 29478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.360509 29478 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:33437
I20260810 22:39:41.360548 29612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:33437 every 8 connection(s)
I20260810 22:39:41.361366 29478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:41.367826 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 29478
I20260810 22:39:41.367980 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:41.368254 20265 external_mini_cluster.cc:1428] Setting key 3a2562661990a664977855c899e8f627
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:41.375258 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:41.363076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:36859 (local address 127.19.202.126:36639)
0810 22:39:41.363208 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:41.363219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:41.363919 (+ 700us) server_negotiation.cc:407] Connection header received
0810 22:39:41.365265 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:41.365268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:41.365329 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:41.365440 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:41.366563 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.367099 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.368321 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.368579 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.368694 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:41.371357 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:41.371361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:41.371364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:41.371396 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:41.373096 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:41.373578 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:41.373584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:41.373588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:41.373681 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:41.374001 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:41.374006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:41.374011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:41.374258 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:41.374361 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:41.374881 (+ 520us) server_negotiation.cc:299] Negotiation successful
0810 22:39:41.375023 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:39:41.376008 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:41.363335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:36859)
0810 22:39:41.363769 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:41.363808 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:41.365045 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:41.365609 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:41.365615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:41.365970 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:41.366415 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:41.366429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.367234 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.367238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:41.368197 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:41.368206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.368728 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.368732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:41.368952 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:41.369527 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:41.369542 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:41.371173 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:41.373212 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:41.373217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:41.373228 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:41.373473 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:41.373773 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:41.373775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:41.373776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:41.373910 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:41.374363 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:41.374368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:41.374581 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 22:39:41.374822 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 22:39:41.375175 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260810 22:39:41.377290 29613 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36639
I20260810 22:39:41.377564 29613 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:41.378186 29613 heartbeater.cc:511] Master 127.19.202.126:36639 requested a full tablet report, sending...
I20260810 22:39:41.379412 29262 ts_manager.cc:194] Registered new tserver with Master: bcf04ae33dbb4680bb38c11ee658e43d (127.19.202.66:33437)
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:39:41.380093 29262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:36859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:39:41.397279 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36639
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:41.506439 29621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:41.506718 29621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:41.506760 29621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:41.510740 29621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:41.510820 29621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:41.510915 29621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:41.516136 29621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:41.517176 29621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:41.518141 29621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:39:41.527612 29621 server_base.cc:1034] running on GCE node
W20260810 22:39:41.527560 29626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.527556 29629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.527560 29627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:41.528198 29621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:41.528823 29621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:41.530023 29621 hybrid_clock.cc:648] HybridClock initialized: now 1786401581530009 us; error 30 us; skew 500 ppm
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:41.532866 29621 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:39:41.533599 29621 webserver.cc:460] Webserver started at http://127.19.202.67:43687/ using document root <none> and password file <none>
I20260810 22:39:41.534153 29621 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:41.534216 29621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:41.534468 29621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:41.536482 29621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "37df956742a540d295a924cbed9afce4"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "511e1e25001d8aa760c05e2768279236"
server_key_iv: "2c5cffe16ce32d79f706bb230aacfbe2"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.537057 29621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "37df956742a540d295a924cbed9afce4"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "511e1e25001d8aa760c05e2768279236"
server_key_iv: "2c5cffe16ce32d79f706bb230aacfbe2"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.540650 29621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:41.543059 29637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.544210 29621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:41.544303 29621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "37df956742a540d295a924cbed9afce4"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "511e1e25001d8aa760c05e2768279236"
server_key_iv: "2c5cffe16ce32d79f706bb230aacfbe2"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.544387 29621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:41.557109 29621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:41.559929 29621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:41.560060 29621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:41.560549 29621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:41.561268 29621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:41.561314 29621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.561348 29621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:41.561367 29621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.569357 29621 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:34629
I20260810 22:39:41.569391 29754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:34629 every 8 connection(s)
I20260810 22:39:41.570178 29621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:41.573035 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 29621
I20260810 22:39:41.573184 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:41.573546 20265 external_mini_cluster.cc:1428] Setting key 7b34340f2a37a08d4aea740d420db81c
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:39:41.583045 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:41.571793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:43305 (local address 127.19.202.126:36639)
0810 22:39:41.571948 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:41.571953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:41.572761 (+ 808us) server_negotiation.cc:407] Connection header received
0810 22:39:41.574333 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:41.574336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:41.574386 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:41.574479 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:41.575893 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.576411 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.577591 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.577803 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.577891 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:41.580236 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:41.580239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:41.580241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:41.580263 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:41.581486 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:41.581895 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:41.581897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:41.581898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:41.581975 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:41.582239 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:41.582241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:41.582242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:41.582389 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:41.582459 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:41.582844 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 22:39:41.582939 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 22:39:41.583742 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:41.572179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:43305)
0810 22:39:41.572619 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:41.572656 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:41.574096 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:41.574672 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:41.574699 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:41.575148 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:41.575733 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:41.575749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.576516 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.576518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:41.577483 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:41.577494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:41.577952 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:41.577954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:41.578087 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:41.578650 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:41.578668 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:41.580095 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:41.581581 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:41.581585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:41.581595 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:41.581812 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:41.582048 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:39:41.582050 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:39:41.582051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:39:41.582167 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:39:41.582469 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:41.582473 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:41.582663 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 22:39:41.582800 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 22:39:41.583028 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
I20260810 22:39:41.584807 29755 heartbeater.cc:347] Connected to a master server at 127.19.202.126:36639
I20260810 22:39:41.585049 29755 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:41.585497 29755 heartbeater.cc:511] Master 127.19.202.126:36639 requested a full tablet report, sending...
I20260810 22:39:41.586674 29262 ts_manager.cc:194] Registered new tserver with Master: 37df956742a540d295a924cbed9afce4 (127.19.202.67:34629)
I20260810 22:39:41.587177 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:39:41.587306 29262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:43305
WARNING: no policy specified for oryx/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab.
Entry for principal oryx/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab.
WARNING: no policy specified for oryx/127.19.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab.
Entry for principal oryx/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab.
WARNING: no policy specified for oryx/127.19.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab.
Entry for principal oryx/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab.
I20260810 22:39:41.653224 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 29331
I20260810 22:39:41.662974 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 29478
I20260810 22:39:41.669608 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 29621
I20260810 22:39:41.677141 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 29229
I20260810 22:39:41.684902 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36639
--webserver_interface=127.19.202.126
--webserver_port=39853
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:36639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:39:41.792904 29765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:41.793160 29765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:41.793200 29765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:41.797125 29765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:41.797176 29765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:41.797195 29765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:41.797212 29765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:41.797247 29765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:41.802299 29765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:36639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:36639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=39853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:41.803267 29765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:41.804221 29765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:41.812605 29771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:41.812646 29765 server_base.cc:1034] running on GCE node
W20260810 22:39:41.812614 29770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.812676 29773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:41.813416 29765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:41.814332 29765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:41.815500 29765 hybrid_clock.cc:648] HybridClock initialized: now 1786401581815482 us; error 29 us; skew 500 ppm
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:41.818270 29765 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:39:41.819017 29765 webserver.cc:460] Webserver started at http://127.19.202.126:39853/ using document root <none> and password file <none>
I20260810 22:39:41.819499 29765 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:41.819566 29765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:41.823103 29765 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:41.825085 29781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:41.826216 29765 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:41.826328 29765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "21739e962b324429bbbaf6b628e40ed0"
format_stamp: "Formatted at 2026-08-10 22:39:40 on dist-test-slave-1x4g"
server_key: "e1de3f0004b6ff980bb4901fdd7616f4"
server_key_iv: "6f48d102ff8e58f7538f6ae5986ef328"
server_key_version: "encryptionkey@0"
I20260810 22:39:41.826855 29765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:41.838500 29765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:41.841040 29765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:41.841173 29765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:41.847932 29765 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:36639
I20260810 22:39:41.847996 29837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:36639 every 8 connection(s)
I20260810 22:39:41.849014 29765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:41.850950 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 29765
I20260810 22:39:41.851367 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:44239
--local_ip_for_outbound_sockets=127.19.202.65
--tserver_master_addrs=127.19.202.126:36639
--webserver_port=45243
--webserver_interface=127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:39:41.853621 29839 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:39:41.855683 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Bootstrap starting.
I20260810 22:39:41.873466 29839 log.cc:824] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:41.879966 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:39:41.880328 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Bootstrap complete.
I20260810 22:39:41.884521 29839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:41.885022 29839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21739e962b324429bbbaf6b628e40ed0, State: Initialized, Role: FOLLOWER
I20260810 22:39:41.885529 29839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:41.885649 29839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:41.885744 29839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:41.885865 29839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:39:41.887668 29839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:41.887995 29839 leader_election.cc:302] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21739e962b324429bbbaf6b628e40ed0; no voters:
I20260810 22:39:41.888322 29839 leader_election.cc:288] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 22:39:41.888414 29843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 22:39:41.888615 29843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [term 2 LEADER]: Becoming Leader. State: Replica: 21739e962b324429bbbaf6b628e40ed0, State: Running, Role: LEADER
I20260810 22:39:41.888933 29843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } }
I20260810 22:39:41.889549 29839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:41.889823 29843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21739e962b324429bbbaf6b628e40ed0. Latest consensus state: current_term: 2 leader_uuid: "21739e962b324429bbbaf6b628e40ed0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } } }
I20260810 22:39:41.889951 29843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:41.890311 29844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "21739e962b324429bbbaf6b628e40ed0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21739e962b324429bbbaf6b628e40ed0" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 36639 } } }
I20260810 22:39:41.890511 29844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0 [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:41.891605 29847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:41.894569 29847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:41.895267 29847 catalog_manager.cc:1259] Loaded cluster ID: 95015009dce04951af8835a3003a04ee
I20260810 22:39:41.895339 29847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:41.897910 29847 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:41.898521 29847 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 21739e962b324429bbbaf6b628e40ed0: Loaded TSK: 0
I20260810 22:39:41.899672 29847 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 22:39:41.963335 29841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:41.963578 29841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:41.963614 29841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:41.967983 29841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:41.968039 29841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:41.968127 29841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:41.973260 29841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.65.keytab
--principal=oryx/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:44239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=45243
--tserver_master_addrs=127.19.202.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:41.974203 29841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:41.975200 29841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:41.984467 29866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.984611 29865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:41.984508 29868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:41.985226 29841 server_base.cc:1034] running on GCE node
I20260810 22:39:41.985683 29841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:41.986408 29841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:41.987640 29841 hybrid_clock.cc:648] HybridClock initialized: now 1786401581987607 us; error 39 us; skew 500 ppm
Aug 10 22:39:41 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401581, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:41.991132 29841 init.cc:377] Logged in from keytab as oryx/127.19.202.65@KRBTEST.COM (short username oryx)
I20260810 22:39:41.992238 29841 webserver.cc:460] Webserver started at http://127.19.202.65:45243/ using document root <none> and password file <none>
I20260810 22:39:41.992940 29841 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:41.993057 29841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:41.998034 29841 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:39:42.000572 29875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.002022 29841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:42.002153 29841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "bec224e543bf4697b8385a9eaf3a8997"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "2105cf02380f516320996a5c1ca9911e"
server_key_iv: "8418d8c918995586f2b64387017fda24"
server_key_version: "encryptionkey@0"
I20260810 22:39:42.002806 29841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:42.020001 29841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:42.023397 29841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:42.023597 29841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:42.024291 29841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:42.025295 29841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:42.025358 29841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.025418 29841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:42.025449 29841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.036628 29841 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:44239
I20260810 22:39:42.036680 30000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:44239 every 8 connection(s)
I20260810 22:39:42.037812 29841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:42.038786 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 29841
I20260810 22:39:42.039289 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:33437
--local_ip_for_outbound_sockets=127.19.202.66
--tserver_master_addrs=127.19.202.126:36639
--webserver_port=43181
--webserver_interface=127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:42.053128 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.040191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:41823)
0810 22:39:42.040850 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.040901 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.042692 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.044034 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.044043 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.044499 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.045137 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.045156 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.047200 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.047202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.048031 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.048040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.049214 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.049217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.049364 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.049950 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.049965 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.052392 (+ 2427us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":134,"threads_started":1}
W20260810 22:39:42.053577 30001 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:36639 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.202.126:36639: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
W20260810 22:39:42.053907 30003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.039982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:41823 (local address 127.19.202.126:36639)
0810 22:39:42.040562 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.040581 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.041009 (+ 428us) server_negotiation.cc:407] Connection header received
0810 22:39:42.043141 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.043157 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.043562 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.043908 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.045338 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.047047 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.048158 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.049063 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.049252 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.052635 (+ 3383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:41823: BlockingRecv error: recv got EOF from 127.19.202.65:41823 (error 108)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":110,"threads_started":1}
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:42.059195 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.054048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:54645)
0810 22:39:42.054213 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.054228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.054302 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.054536 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.054538 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.054670 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.054777 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.054782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.055775 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.055777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.056289 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.056296 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.056888 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.056891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.056960 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.057421 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.057432 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.059118 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97}
W20260810 22:39:42.059427 30003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.054111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:54645 (local address 127.19.202.126:36639)
0810 22:39:42.054298 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.054302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.054317 (+ 15us) server_negotiation.cc:407] Connection header received
0810 22:39:42.054377 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.054380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.054420 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.054501 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.054878 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.055678 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.056391 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.056756 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.056888 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.059286 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:54645: BlockingRecv error: recv got EOF from 127.19.202.65:54645 (error 108)
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":120}
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:42.064343 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.059709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:42067)
0810 22:39:42.059789 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.059803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.059867 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.060063 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.060066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.060173 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.060233 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.060236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.061149 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.061152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.061626 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.061633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.062160 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.062162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.062216 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.062648 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.062658 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.064267 (+ 1609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 22:39:42.064499 30003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.059772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:42067 (local address 127.19.202.126:36639)
0810 22:39:42.059857 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.059861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.059873 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:39:42.059936 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.059938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.059971 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.060041 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.060323 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.061057 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.061731 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.062060 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.062158 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.064407 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:42067: BlockingRecv error: recv got EOF from 127.19.202.65:42067 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 22:39:42.064531 30001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:39:42.152398 30005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:42.152634 30005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:42.152669 30005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:42.156419 30005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:42.156461 30005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:42.156513 30005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:42.161279 30005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.66.keytab
--principal=oryx/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:33437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=43181
--tserver_master_addrs=127.19.202.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:42.162149 30005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:42.163100 30005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:39:42.172384 30005 server_base.cc:1034] running on GCE node
W20260810 22:39:42.172355 30012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:42.172335 30011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:42.172322 30015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:42.173013 30005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:42.173561 30005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:42.174760 30005 hybrid_clock.cc:648] HybridClock initialized: now 1786401582174747 us; error 32 us; skew 500 ppm
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401582, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:42.177434 30005 init.cc:377] Logged in from keytab as oryx/127.19.202.66@KRBTEST.COM (short username oryx)
I20260810 22:39:42.178222 30005 webserver.cc:460] Webserver started at http://127.19.202.66:43181/ using document root <none> and password file <none>
I20260810 22:39:42.178742 30005 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:42.178857 30005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:42.182487 30005 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:39:42.184356 30022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.185616 30005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:42.185765 30005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "bcf04ae33dbb4680bb38c11ee658e43d"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "100f484c33ba8c4ebd527fe2b3c2dc0d"
server_key_iv: "9bbd25ba7387dc4190d358efa250cc36"
server_key_version: "encryptionkey@0"
I20260810 22:39:42.186311 30005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:42.197916 30005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:42.200317 30005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:42.200498 30005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:42.201074 30005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:42.201798 30005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:42.201879 30005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.201965 30005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:42.202015 30005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.211190 30005 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:33437
I20260810 22:39:42.211257 30142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:33437 every 8 connection(s)
I20260810 22:39:42.212110 30005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:42.217540 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30005
I20260810 22:39:42.217985 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:34629
--local_ip_for_outbound_sockets=127.19.202.67
--tserver_master_addrs=127.19.202.126:36639
--webserver_port=43687
--webserver_interface=127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:39:42.223062 30003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.215051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:58261 (local address 127.19.202.126:36639)
0810 22:39:42.215225 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.215231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.215252 (+ 21us) server_negotiation.cc:407] Connection header received
0810 22:39:42.216169 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.216172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.216232 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.216305 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.217545 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.218176 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.219289 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.219521 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.219624 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.222893 (+ 3269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:58261: BlockingRecv error: recv got EOF from 127.19.202.66:58261 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 22:39:42.223490 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.214267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:58261)
0810 22:39:42.214695 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.214750 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.215963 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.216485 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.216491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.216950 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.217406 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.217420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.218315 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.218318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.219164 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.219173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.219658 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.219661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.219822 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.220327 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.220341 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.222705 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":124,"threads_started":1}
W20260810 22:39:42.224021 30143 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:36639 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.202.126:36639: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:42.229112 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.224430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:59359)
0810 22:39:42.224532 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.224545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.224612 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.224887 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.224890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.225048 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.225128 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.225133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.225821 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.225826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.226336 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.226344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.226783 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.226787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.226879 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.227313 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.227324 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.228991 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 22:39:42.229202 30003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.224528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:59359 (local address 127.19.202.126:36639)
0810 22:39:42.224630 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.224636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.224651 (+ 15us) server_negotiation.cc:407] Connection header received
0810 22:39:42.224685 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.224688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.224747 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.224873 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.225240 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.225714 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.226462 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.226657 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.226773 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.229098 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:59359: BlockingRecv error: recv got EOF from 127.19.202.66:59359 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:42.234398 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.229715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:59181)
0810 22:39:42.229808 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.229821 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.229887 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.230086 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.230089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.230231 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.230306 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.230311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.230972 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.230975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.231798 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.231807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.232190 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.232194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.232273 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.232640 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.232652 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.234280 (+ 1628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 22:39:42.234571 30003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.229761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:59181 (local address 127.19.202.126:36639)
0810 22:39:42.229856 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.229861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.229878 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:39:42.229956 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.229959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.229993 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.230053 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.230407 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.230866 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.231902 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.232095 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.232166 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.234380 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:59181: BlockingRecv error: recv got EOF from 127.19.202.66:59181 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 22:39:42.234756 30143 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:39:42.326823 30146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:42.327082 30146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:42.327153 30146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:42.331200 30146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:42.331280 30146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:42.331375 30146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:42.336342 30146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/krb5kdc/oryx_127.19.202.67.keytab
--principal=oryx/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:34629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=43687
--tserver_master_addrs=127.19.202.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:42.337292 30146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:42.338241 30146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:42.348320 30155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:42.348291 30153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:42.348326 30152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:42.348520 30146 server_base.cc:1034] running on GCE node
I20260810 22:39:42.348953 30146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:42.349543 30146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:42.350724 30146 hybrid_clock.cc:648] HybridClock initialized: now 1786401582350710 us; error 23 us; skew 500 ppm
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401582, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:39:42.353363 30146 init.cc:377] Logged in from keytab as oryx/127.19.202.67@KRBTEST.COM (short username oryx)
I20260810 22:39:42.354233 30146 webserver.cc:460] Webserver started at http://127.19.202.67:43687/ using document root <none> and password file <none>
I20260810 22:39:42.354777 30146 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:42.354920 30146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:42.358632 30146 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260810 22:39:42.360620 30162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.361780 30146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:42.361923 30146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "37df956742a540d295a924cbed9afce4"
format_stamp: "Formatted at 2026-08-10 22:39:41 on dist-test-slave-1x4g"
server_key: "511e1e25001d8aa760c05e2768279236"
server_key_iv: "2c5cffe16ce32d79f706bb230aacfbe2"
server_key_version: "encryptionkey@0"
I20260810 22:39:42.362447 30146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:42.377610 30146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:42.380920 30146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:42.381145 30146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:42.381749 30146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:42.382566 30146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:42.382637 30146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.382706 30146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:42.382756 30146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:42.392400 30146 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:34629
I20260810 22:39:42.392437 30283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:34629 every 8 connection(s)
I20260810 22:39:42.393395 30146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:42.395684 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30146
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:39:42.405257 30003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.395071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:42811 (local address 127.19.202.126:36639)
0810 22:39:42.395198 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.395202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.396073 (+ 871us) server_negotiation.cc:407] Connection header received
0810 22:39:42.397921 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.397923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.397952 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.397998 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.399238 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.399732 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.401322 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.401558 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.401660 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.405131 (+ 3471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:42811: BlockingRecv error: recv got EOF from 127.19.202.67:42811 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:39:42.405901 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.395466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:42811)
0810 22:39:42.395935 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.395982 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.397682 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.398169 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.398175 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.398506 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.399067 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.399084 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.399844 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.399846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.401203 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.401215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.401721 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.401725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.401871 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.402469 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.402484 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.404991 (+ 2507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":138,"threads_started":1}
I20260810 22:39:42.406100 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.396676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.19.202.126:36639)
0810 22:39:42.396905 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.396908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.396922 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:39:42.397042 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.397044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.397083 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.397162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.398103 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.398631 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.399619 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.399907 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.399974 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.401618 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:42.401626 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:42.401645 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:39:42.401676 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:42.403675 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:42.404184 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:42.404188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:42.404193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:42.404314 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:39:42.404629 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:39:42.404631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:39:42.404632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:39:42.405064 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:42.405249 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:42.405615 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 22:39:42.405876 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":78,"threads_started":1}
W20260810 22:39:42.406620 30284 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:36639 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.202.126:36639: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:42.412002 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.407027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:42429)
0810 22:39:42.407205 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.407220 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.407287 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.407503 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.407506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.407656 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.407758 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.407765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.408666 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.408670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.409426 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.409436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.409906 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.409909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.410004 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.410371 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.410381 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.411926 (+ 1545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112}
W20260810 22:39:42.412178 30289 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.407109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:42429 (local address 127.19.202.126:36639)
0810 22:39:42.407246 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.407249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.407262 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:42.407366 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.407368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.407398 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.407462 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.407864 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.408561 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.409547 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.409781 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.409893 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.412064 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:42429: BlockingRecv error: recv got EOF from 127.19.202.67:42429 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:42 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:42.416668 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:42.412488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:54715)
0810 22:39:42.412590 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:42.412603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:42.412668 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:42.412940 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:42.412942 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:42.413056 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:42.413117 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.413121 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.413716 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.413718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.414187 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:42.414193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.414556 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.414558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:42.414631 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.415063 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:42.415075 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:42.416562 (+ 1487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 22:39:42.416826 30289 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:42.412534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:54715 (local address 127.19.202.126:36639)
0810 22:39:42.412658 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:42.412661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:42.412673 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:39:42.412749 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:42.412751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:42.412826 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:42.412907 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:42.413206 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.413632 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.414281 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:42.414469 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:42.414539 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:42.416697 (+ 2158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:54715: BlockingRecv error: recv got EOF from 127.19.202.67:54715 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 22:39:42.416985 30284 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 22:39:43 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:43 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:43.071009 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:43.065277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:55927)
0810 22:39:43.065616 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:43.065630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:43.065779 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:43.066037 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:43.066040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:43.066210 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:43.066310 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:43.066318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.067121 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.067123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:43.067669 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:43.067676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.068293 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.068298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:43.068387 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:43.069090 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:43.069106 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:43.070890 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":81,"threads_started":1}
W20260810 22:39:43.071187 30292 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:43.065408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:55927 (local address 127.19.202.126:36639)
0810 22:39:43.065658 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:43.065662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:43.065748 (+ 86us) server_negotiation.cc:407] Connection header received
0810 22:39:43.065862 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:43.065865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:43.065912 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:43.066003 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:43.066415 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.067024 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.067812 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.068166 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.068268 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:43.071024 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:55927: BlockingRecv error: recv got EOF from 127.19.202.65:55927 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":109,"threads_started":1}
Aug 10 22:39:43 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:43 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:43.240878 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:43.235338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:34153)
0810 22:39:43.235614 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:43.235629 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:43.235729 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:43.235973 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:43.235976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:43.236156 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:43.236256 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:43.236261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.237081 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.237084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:43.237666 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:43.237675 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.238097 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.238100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:43.238167 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:43.238793 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:43.238804 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:43.240711 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
W20260810 22:39:43.241218 30292 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:43.235415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:34153 (local address 127.19.202.126:36639)
0810 22:39:43.235550 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:43.235554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:43.235699 (+ 145us) server_negotiation.cc:407] Connection header received
0810 22:39:43.235807 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:43.235809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:43.235849 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:43.235925 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:43.236357 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.236980 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.237776 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.237991 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.238065 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:43.240945 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:34153: BlockingRecv error: recv got EOF from 127.19.202.66:34153 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 22:39:43 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:43 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:43.422938 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:43.417554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:40727)
0810 22:39:43.417822 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:43.417835 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:43.417946 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:43.418179 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:43.418181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:43.418351 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:43.418451 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:43.418456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.419132 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.419134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:43.419663 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:43.419670 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.420068 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.420070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:43.420128 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:43.420609 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:43.420620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:43.422750 (+ 2130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
W20260810 22:39:43.422981 30292 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:43.417647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:40727 (local address 127.19.202.126:36639)
0810 22:39:43.417781 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:43.417785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:43.417923 (+ 138us) server_negotiation.cc:407] Connection header received
0810 22:39:43.418013 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:43.418015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:43.418057 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:43.418125 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:43.418546 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.419036 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.419756 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:43.419970 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:43.420041 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:43.422893 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:40727: BlockingRecv error: recv got EOF from 127.19.202.67:40727 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 22:39:44 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:44 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:44.077199 30296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:44.071864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:43317)
0810 22:39:44.072195 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:44.072209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:44.072337 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:44.072639 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:44.072641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:44.072846 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:44.072962 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:44.072968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.073668 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.073670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:44.074212 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:44.074219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.074599 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.074602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:44.074660 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:44.075162 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:44.075173 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:44.077063 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":87,"threads_started":1}
W20260810 22:39:44.077344 30295 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:44.071900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:43317 (local address 127.19.202.126:36639)
0810 22:39:44.072162 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:44.072166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:44.072309 (+ 143us) server_negotiation.cc:407] Connection header received
0810 22:39:44.072446 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:44.072451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:44.072509 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:44.072604 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:44.073062 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.073577 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.074307 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.074509 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.074578 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:44.077210 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:43317: BlockingRecv error: recv got EOF from 127.19.202.65:43317 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
Aug 10 22:39:44 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:44 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:44.246842 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:44.241733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:54871)
0810 22:39:44.242006 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:44.242022 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:44.242168 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:44.242387 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:44.242390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:44.242564 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:44.242661 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:44.242665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.243351 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.243354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:44.243864 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:44.243871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.244238 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.244240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:44.244299 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:44.244805 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:44.244816 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:44.246701 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
W20260810 22:39:44.246928 30295 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:44.241837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:54871 (local address 127.19.202.126:36639)
0810 22:39:44.241996 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:44.242002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:44.242113 (+ 111us) server_negotiation.cc:407] Connection header received
0810 22:39:44.242237 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:44.242239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:44.242279 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:44.242341 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:44.242757 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.243263 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.243959 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.244149 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.244218 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:44.246835 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:54871: BlockingRecv error: recv got EOF from 127.19.202.66:54871 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 22:39:44 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:44 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:44.429138 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:44.423824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:44537)
0810 22:39:44.424090 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:44.424103 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:44.424185 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:44.424427 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:44.424429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:44.424593 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:44.424686 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:44.424690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.425457 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.425459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:44.426021 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:44.426029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.426460 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.426463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:44.426522 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:44.427101 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:44.427112 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:44.429007 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
W20260810 22:39:44.429281 30295 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:44.423980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:44537 (local address 127.19.202.126:36639)
0810 22:39:44.424133 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:44.424136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:44.424150 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:39:44.424271 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:44.424273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:44.424320 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:44.424393 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:44.424793 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.425353 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.426139 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:44.426359 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:44.426442 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:44.429136 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:44537: BlockingRecv error: recv got EOF from 127.19.202.67:44537 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 22:39:45 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:45 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:45.083197 30300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:45.078014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:56923)
0810 22:39:45.078303 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:45.078318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:45.078429 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:45.078681 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:45.078683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:45.078863 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:45.078977 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:45.078982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.079723 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.079725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:45.080275 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:45.080282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.080688 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.080691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:45.080752 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:45.081259 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:45.081270 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:45.083084 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":80,"threads_started":1}
W20260810 22:39:45.083359 30301 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:45.078100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:56923 (local address 127.19.202.126:36639)
0810 22:39:45.078385 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:45.078388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:45.078402 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:39:45.078517 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:45.078520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:45.078564 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:45.078646 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:45.079087 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.079625 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.080374 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.080591 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.080672 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:45.083222 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:56923: BlockingRecv error: recv got EOF from 127.19.202.65:56923 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
Aug 10 22:39:45 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:45 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:45.253085 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:45.247643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:51573)
0810 22:39:45.247928 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:45.247940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:45.248097 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:45.248391 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:45.248394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:45.248609 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:45.248707 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:45.248712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.249528 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.249533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:45.250064 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:45.250071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.250545 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.250548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:45.250637 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:45.251110 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:45.251121 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:45.252975 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":109,"threads_started":1}
W20260810 22:39:45.253227 30301 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:45.247700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:51573 (local address 127.19.202.126:36639)
0810 22:39:45.247865 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:45.247870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:45.248046 (+ 176us) server_negotiation.cc:407] Connection header received
0810 22:39:45.248225 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:45.248229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:45.248279 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:45.248370 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:45.248863 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.249412 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.250179 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.250433 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.250541 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:45.253100 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:51573: BlockingRecv error: recv got EOF from 127.19.202.66:51573 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 22:39:45 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:45 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:45.435375 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:45.429877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:45683)
0810 22:39:45.430160 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:45.430174 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:45.430274 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:45.430522 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:45.430524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:45.430736 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:45.430863 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:45.430869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.431676 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.431679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:45.432246 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:45.432255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.432662 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.432665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:45.432740 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:45.433332 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:45.433344 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:45.435244 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":116,"threads_started":1}
W20260810 22:39:45.435510 30301 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:45.429919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:45683 (local address 127.19.202.126:36639)
0810 22:39:45.430055 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:45.430059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:45.430236 (+ 177us) server_negotiation.cc:407] Connection header received
0810 22:39:45.430367 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:45.430370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:45.430422 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:45.430485 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:45.430981 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.431555 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.432353 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:45.432565 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:45.432637 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:45.435368 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:45683: BlockingRecv error: recv got EOF from 127.19.202.67:45683 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 22:39:46 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:46 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:46.089956 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:46.084031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:58469)
0810 22:39:46.084332 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:46.084348 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:46.084456 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:46.084806 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:46.084809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:46.085058 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:46.085165 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:46.085194 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.086063 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.086066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:46.086634 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:46.086642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.087055 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.087058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:46.087124 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:46.087710 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:46.087724 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:46.089749 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":107,"threads_started":1}
W20260810 22:39:46.090178 30305 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:46.084148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:58469 (local address 127.19.202.126:36639)
0810 22:39:46.084504 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:46.084509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:46.084525 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:39:46.084570 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:46.084574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:46.084635 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:46.084749 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:46.085299 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.085956 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.086739 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.086954 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.087027 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:46.089927 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:58469: BlockingRecv error: recv got EOF from 127.19.202.65:58469 (error 108)
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
Aug 10 22:39:46 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:46 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:46.264206 30306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:46.254034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:52375)
0810 22:39:46.254368 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:46.254391 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:46.254540 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:46.254837 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:46.254840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:46.255077 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:46.255181 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:46.255188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.256173 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.256180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:46.256769 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:46.256778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.257386 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.257390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:46.257487 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:46.262242 (+ 4755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:46.262255 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:46.264096 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
W20260810 22:39:46.264384 30305 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:46.254118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:52375 (local address 127.19.202.126:36639)
0810 22:39:46.254336 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:46.254341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:46.254474 (+ 133us) server_negotiation.cc:407] Connection header received
0810 22:39:46.254645 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:46.254649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:46.254704 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:46.254792 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:46.255328 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.256042 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.256985 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.257235 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.257378 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:46.264248 (+ 6870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:52375: BlockingRecv error: recv got EOF from 127.19.202.66:52375 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 10 22:39:46 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:46 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:46.442052 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:46.436242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:52573)
0810 22:39:46.436516 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:46.436529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:46.436639 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:46.436935 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:46.436938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:46.437123 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:46.437226 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:46.437230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.437989 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.437992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:46.438542 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:46.438550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.439106 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.439110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:46.439214 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:46.439805 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:46.439816 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:46.441864 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
W20260810 22:39:46.442113 30305 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:46.436319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:52573 (local address 127.19.202.126:36639)
0810 22:39:46.436461 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:46.436464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:46.436609 (+ 145us) server_negotiation.cc:407] Connection header received
0810 22:39:46.436719 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:46.436722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:46.436767 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:46.436910 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:46.437339 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.437862 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.438682 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:46.438991 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:46.439103 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:46.441999 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:52573: BlockingRecv error: recv got EOF from 127.19.202.67:52573 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 22:39:47 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:47 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:47.096266 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:47.090766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:35795)
0810 22:39:47.091063 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:47.091077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:47.091168 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:47.091431 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:47.091433 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:47.091636 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:47.091735 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:47.091740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.092504 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.092507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:47.093169 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:47.093179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.093607 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.093610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:47.093675 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:47.094274 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:47.094287 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:47.096151 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":109,"threads_started":1}
W20260810 22:39:47.096446 30310 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:47.090850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:35795 (local address 127.19.202.126:36639)
0810 22:39:47.091100 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:47.091103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:47.091120 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:39:47.091264 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:47.091266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:47.091317 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:47.091387 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:47.091838 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.092389 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.093281 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.093508 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.093583 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:47.096293 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:35795: BlockingRecv error: recv got EOF from 127.19.202.65:35795 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
Aug 10 22:39:47 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:47 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:47.270788 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:47.265162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:50535)
0810 22:39:47.265636 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:47.265651 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:47.265750 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:47.265984 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:47.265986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:47.266172 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:47.266281 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:47.266286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.266967 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.266970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:47.267546 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:47.267554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.268061 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.268064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:47.268162 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:47.268851 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:47.268867 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:47.270680 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":107,"threads_started":1}
W20260810 22:39:47.270931 30310 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:47.265287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:50535 (local address 127.19.202.126:36639)
0810 22:39:47.265451 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:47.265455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:47.265724 (+ 269us) server_negotiation.cc:407] Connection header received
0810 22:39:47.265831 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:47.265833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:47.265876 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:47.265943 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:47.266388 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.266869 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.267648 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.267933 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.268040 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:47.270813 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:50535: BlockingRecv error: recv got EOF from 127.19.202.66:50535 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 22:39:47 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:47 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:47.448143 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:47.442922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:37519)
0810 22:39:47.443271 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:47.443285 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:47.443390 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:47.443618 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:47.443620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:47.443808 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:47.443916 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:47.443921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.444575 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.444577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:47.445196 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:47.445204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.445647 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.445650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:47.445721 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:47.446229 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:47.446241 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:47.448042 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":90,"threads_started":1}
W20260810 22:39:47.448295 30310 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:47.443136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:37519 (local address 127.19.202.126:36639)
0810 22:39:47.443273 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:47.443277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:47.443342 (+ 65us) server_negotiation.cc:407] Connection header received
0810 22:39:47.443471 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:47.443473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:47.443515 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:47.443574 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:47.444012 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.444487 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.445303 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:47.445523 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:47.445602 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:47.448155 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:37519: BlockingRecv error: recv got EOF from 127.19.202.67:37519 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 22:39:48 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:48 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:48.103179 30313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:48.097144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:53777)
0810 22:39:48.097467 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:48.097481 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:48.097589 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:48.097889 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:48.097891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:48.098175 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:48.098283 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:48.098292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.099227 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.099231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:48.099812 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:48.099823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.100292 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.100295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:48.100373 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:48.100983 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:48.101009 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:48.103009 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":110,"threads_started":1}
W20260810 22:39:48.103255 30314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:48.097216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:53777 (local address 127.19.202.126:36639)
0810 22:39:48.097455 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:48.097459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:48.097552 (+ 93us) server_negotiation.cc:407] Connection header received
0810 22:39:48.097671 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:48.097673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:48.097716 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:48.097850 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:48.098417 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.099109 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.099941 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.100186 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.100268 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:48.103132 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:53777: BlockingRecv error: recv got EOF from 127.19.202.65:53777 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
Aug 10 22:39:48 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:48 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:48.277299 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:48.271720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:57345)
0810 22:39:48.272009 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:48.272022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:48.272155 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:48.272430 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:48.272433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:48.272649 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:48.272741 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:48.272746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.273545 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.273550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:48.274095 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:48.274102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.274575 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.274579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:48.274662 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:48.275133 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:48.275144 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:48.277129 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
W20260810 22:39:48.277364 30314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:48.271793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:57345 (local address 127.19.202.126:36639)
0810 22:39:48.271962 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:48.271966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:48.272095 (+ 129us) server_negotiation.cc:407] Connection header received
0810 22:39:48.272253 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:48.272257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:48.272311 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:48.272397 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:48.272904 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.273434 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.274215 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.274447 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.274571 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:48.277250 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:57345: BlockingRecv error: recv got EOF from 127.19.202.66:57345 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 22:39:48 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:48 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:48.454210 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:48.448896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:56221)
0810 22:39:48.449144 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:48.449157 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:48.449252 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:48.449474 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:48.449476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:48.449663 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:48.449779 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:48.449784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.450445 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.450448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:48.450953 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:48.450960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.451558 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.451561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:48.451643 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:48.452272 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:48.452283 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:48.454103 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
W20260810 22:39:48.454360 30314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:48.448990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:56221 (local address 127.19.202.126:36639)
0810 22:39:48.449141 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:48.449145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:48.449222 (+ 77us) server_negotiation.cc:407] Connection header received
0810 22:39:48.449325 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:48.449327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:48.449369 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:48.449436 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:48.449871 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.450357 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.451094 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:48.451441 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:48.451535 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:48.454218 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:56221: BlockingRecv error: recv got EOF from 127.19.202.67:56221 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 22:39:49 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:49 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:49.109323 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:49.103933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:56363)
0810 22:39:49.104316 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:49.104328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:49.104433 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:49.104667 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:49.104669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:49.104877 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:49.104975 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:49.104980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.105729 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.105731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:49.106290 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:49.106297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.106736 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.106738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:49.106797 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:49.107275 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:49.107286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:49.109163 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":94,"threads_started":1}
W20260810 22:39:49.109416 30318 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:49.104031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:56363 (local address 127.19.202.126:36639)
0810 22:39:49.104328 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:49.104332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:49.104403 (+ 71us) server_negotiation.cc:407] Connection header received
0810 22:39:49.104512 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:49.104514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:49.104556 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:49.104630 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:49.105072 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.105633 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.106410 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.106630 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.106719 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:49.109282 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:56363: BlockingRecv error: recv got EOF from 127.19.202.65:56363 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":110,"threads_started":1}
Aug 10 22:39:49 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:49 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:49.283921 30319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:49.278228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:44179)
0810 22:39:49.278560 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:49.278573 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:49.278667 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:49.278937 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:49.278939 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:49.279123 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:49.279215 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:49.279220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.280244 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.280250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:49.280970 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:49.280981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.281387 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.281390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:49.281453 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:49.281936 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:49.281946 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:49.283798 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
W20260810 22:39:49.284162 30318 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:49.278415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:44179 (local address 127.19.202.126:36639)
0810 22:39:49.278555 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:49.278559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:49.278639 (+ 80us) server_negotiation.cc:407] Connection header received
0810 22:39:49.278755 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:49.278757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:49.278797 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:49.278902 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:49.279322 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.280125 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.281074 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.281282 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.281377 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:49.284047 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:44179: BlockingRecv error: recv got EOF from 127.19.202.66:44179 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 22:39:49 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:49 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:49.459906 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:49.454939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:48541)
0810 22:39:49.455240 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:49.455270 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:49.455367 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:49.455591 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:49.455593 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:49.455775 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:49.455871 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:49.455875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.456582 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.456585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:49.457155 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:49.457162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.457568 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.457571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:49.457631 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:49.458118 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:49.458128 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:49.459802 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
W20260810 22:39:49.460029 30318 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:49.455026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:48541 (local address 127.19.202.126:36639)
0810 22:39:49.455157 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:49.455161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:49.455338 (+ 177us) server_negotiation.cc:407] Connection header received
0810 22:39:49.455444 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:49.455446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:49.455491 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:49.455549 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:49.455968 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.456481 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.457252 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:49.457473 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:49.457548 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:49.459923 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:48541: BlockingRecv error: recv got EOF from 127.19.202.67:48541 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 22:39:50 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:50 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:50.115387 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:50.110168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:37545)
0810 22:39:50.110504 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:50.110519 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:50.110611 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:50.110871 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:50.110873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:50.111055 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:50.111151 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:50.111156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.111856 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.111859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:50.112395 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:50.112402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.112841 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.112844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:50.112907 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:50.113385 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:50.113396 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:50.115255 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":90,"threads_started":1}
W20260810 22:39:50.115571 30322 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:50.110283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:37545 (local address 127.19.202.126:36639)
0810 22:39:50.110560 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:50.110563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:50.110576 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:39:50.110712 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:50.110715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:50.110756 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:50.110831 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:50.111248 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.111766 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.112499 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.112715 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.112865 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:50.115429 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:37545: BlockingRecv error: recv got EOF from 127.19.202.65:37545 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
Aug 10 22:39:50 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:50 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:50.290647 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:50.284729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:57095)
0810 22:39:50.285081 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:50.285104 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:50.285215 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:50.285486 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:50.285489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:50.285688 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:50.285794 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:50.285799 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.286707 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.286710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:50.287309 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:50.287319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.287748 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.287751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:50.287811 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:50.288431 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:50.288444 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:50.290480 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":166,"threads_started":1}
W20260810 22:39:50.290723 30322 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:50.284973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:57095 (local address 127.19.202.126:36639)
0810 22:39:50.285107 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:50.285117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:50.285179 (+ 62us) server_negotiation.cc:407] Connection header received
0810 22:39:50.285299 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:50.285301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:50.285362 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:50.285441 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:50.285931 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.286592 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.287419 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.287636 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.287704 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:50.290593 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:57095: BlockingRecv error: recv got EOF from 127.19.202.66:57095 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 22:39:50 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:50 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:50.466243 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:50.460726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:53475)
0810 22:39:50.461017 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:50.461030 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:50.461120 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:50.461358 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:50.461364 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:50.461592 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:50.461691 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:50.461697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.462546 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.462551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:50.463113 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:50.463122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.463628 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.463631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:50.463715 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:50.464268 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:50.464278 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:50.466130 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":119,"threads_started":1}
W20260810 22:39:50.466581 30322 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:50.460877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:53475 (local address 127.19.202.126:36639)
0810 22:39:50.461007 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:50.461017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:50.461095 (+ 78us) server_negotiation.cc:407] Connection header received
0810 22:39:50.461197 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:50.461200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:50.461242 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:50.461301 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:50.461815 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.462437 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.463243 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:50.463487 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:50.463622 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:50.466454 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:53475: BlockingRecv error: recv got EOF from 127.19.202.67:53475 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 22:39:51 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:51 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:51.121678 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:51.116247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:36589)
0810 22:39:51.116540 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:51.116554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:51.116661 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:51.116944 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:51.116947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:51.117136 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:51.117239 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:51.117244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.118006 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.118009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:51.118557 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:51.118564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.118981 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.118983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:51.119047 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:51.119541 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:51.119552 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:51.121541 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
W20260810 22:39:51.121809 30326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:51.116316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:36589 (local address 127.19.202.126:36639)
0810 22:39:51.116581 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:51.116585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:51.116615 (+ 30us) server_negotiation.cc:407] Connection header received
0810 22:39:51.116746 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:51.116748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:51.116810 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:51.116894 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:51.117340 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.117891 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.118655 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.118879 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.118955 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:51.121671 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:36589: BlockingRecv error: recv got EOF from 127.19.202.65:36589 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
Aug 10 22:39:51 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:51 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:39:51.303644 30326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:51.291569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:48213 (local address 127.19.202.126:36639)
0810 22:39:51.291726 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:51.291730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:51.291961 (+ 231us) server_negotiation.cc:407] Connection header received
0810 22:39:51.292206 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:51.292210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:51.292268 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:51.292361 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:51.292849 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.293433 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.294257 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.294498 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.294603 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:51.303552 (+ 8949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:48213: BlockingRecv error: recv got EOF from 127.19.202.66:48213 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 22:39:51.303673 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:51.291518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:48213)
0810 22:39:51.291854 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:51.291866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:51.292064 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:51.292395 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:51.292398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:51.292604 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:51.292702 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:51.292708 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.293557 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.293582 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:51.294123 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:51.294131 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.294622 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.294626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:51.294719 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:51.300959 (+ 6240us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:51.300971 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:51.303437 (+ 2466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":110,"threads_started":1}
Aug 10 22:39:51 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:51 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:51.472702 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:51.467122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:44231)
0810 22:39:51.467412 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:51.467426 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:51.467560 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:51.467832 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:51.467835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:51.468059 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:51.468157 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:51.468163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.469012 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.469017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:51.469566 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:51.469575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.470033 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.470037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:51.470120 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:51.470576 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:51.470587 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:51.472513 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":93,"threads_started":1}
W20260810 22:39:51.472733 30326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:51.467185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:44231 (local address 127.19.202.126:36639)
0810 22:39:51.467348 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:51.467353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:51.467512 (+ 159us) server_negotiation.cc:407] Connection header received
0810 22:39:51.467666 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:51.467670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:51.467718 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:51.467810 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:51.468283 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.468895 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.469688 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:51.469921 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:51.470028 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:51.472630 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:44231: BlockingRecv error: recv got EOF from 127.19.202.67:44231 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 22:39:52 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:52 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:52.127645 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:52.122570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:48221)
0810 22:39:52.122836 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:52.122848 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:52.122943 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:52.123163 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:52.123165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:52.123349 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:52.123459 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:52.123465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.124194 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.124196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:52.124729 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:52.124736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.125199 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.125201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:52.125264 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:52.125772 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:52.125784 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:52.127535 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
W20260810 22:39:52.127822 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:52.122614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:48221 (local address 127.19.202.126:36639)
0810 22:39:52.122837 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:52.122840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:52.122920 (+ 80us) server_negotiation.cc:407] Connection header received
0810 22:39:52.123016 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:52.123018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:52.123058 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:52.123127 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:52.123557 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.124105 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.124879 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.125103 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.125177 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:52.127661 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:48221: BlockingRecv error: recv got EOF from 127.19.202.65:48221 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
Aug 10 22:39:52 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:52 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:52.309661 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:52.304383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:48005)
0810 22:39:52.304651 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:52.304663 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:52.304752 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:52.305060 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:52.305062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:52.305239 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:52.305334 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:52.305338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.306112 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.306115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:52.306666 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:52.306673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.307074 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.307076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:52.307170 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:52.307650 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:52.307661 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:52.309549 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
W20260810 22:39:52.309789 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:52.304452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:48005 (local address 127.19.202.126:36639)
0810 22:39:52.304616 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:52.304620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:52.304731 (+ 111us) server_negotiation.cc:407] Connection header received
0810 22:39:52.304900 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:52.304903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:52.304975 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:52.305038 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:52.305433 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.305989 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.306759 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.306971 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.307046 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:52.309674 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:48005: BlockingRecv error: recv got EOF from 127.19.202.66:48005 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 22:39:52 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:52 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:52.478690 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:52.473629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:57345)
0810 22:39:52.473920 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:52.473936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:52.474045 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:52.474292 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:52.474294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:52.474476 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:52.474590 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:52.474595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.475281 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.475283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:52.475847 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:52.475854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.476259 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.476262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:52.476323 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:52.476871 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:52.476886 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:52.478568 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":97,"threads_started":1}
W20260810 22:39:52.478858 30331 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:52.473738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:57345 (local address 127.19.202.126:36639)
0810 22:39:52.473913 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:52.473917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:52.473993 (+ 76us) server_negotiation.cc:407] Connection header received
0810 22:39:52.474137 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:52.474139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:52.474186 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:52.474252 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:52.474687 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.475194 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.475956 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:52.476167 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:52.476239 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:52.478705 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:57345: BlockingRecv error: recv got EOF from 127.19.202.67:57345 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 22:39:53 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:53 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:53.135203 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:53.128527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:36561)
0810 22:39:53.128866 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:53.128883 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:53.129003 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:53.129349 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:53.129366 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:53.129610 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:53.129775 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:53.129785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.130902 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.130906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:53.131600 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:53.131609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.132157 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.132161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:53.132256 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:53.132837 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:53.132849 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:53.135044 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
W20260810 22:39:53.135262 30335 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:53.128578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:36561 (local address 127.19.202.126:36639)
0810 22:39:53.128853 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:53.128859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:53.128943 (+ 84us) server_negotiation.cc:407] Connection header received
0810 22:39:53.129128 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:53.129131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:53.129203 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:53.129311 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:53.129897 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.130773 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.131709 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.132058 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.132142 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:53.135153 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:36561: BlockingRecv error: recv got EOF from 127.19.202.65:36561 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":91,"threads_started":1}
Aug 10 22:39:53 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:53 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:53.316651 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:53.310541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:34075)
0810 22:39:53.310854 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:53.310867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:53.310970 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:53.311207 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:53.311209 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:53.311402 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:53.311498 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:53.311503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.312500 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.312504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:53.313330 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:53.313339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.313989 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.313992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:53.314054 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:53.314616 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:53.314630 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:53.316480 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":131,"threads_started":1}
W20260810 22:39:53.316720 30335 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:53.310588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:34075 (local address 127.19.202.126:36639)
0810 22:39:53.310749 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:53.310755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:53.310947 (+ 192us) server_negotiation.cc:407] Connection header received
0810 22:39:53.311057 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:53.311059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:53.311106 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:53.311173 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:53.311625 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.312367 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.313465 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.313850 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.313979 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:53.316612 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:34075: BlockingRecv error: recv got EOF from 127.19.202.66:34075 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 22:39:53 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:53 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:53.484974 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:53.479513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:60101)
0810 22:39:53.479788 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:53.479812 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:53.479912 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:53.480126 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:53.480128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:53.480300 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:53.480398 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:53.480403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.481167 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.481170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:53.481671 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:53.481677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.482062 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.482064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:53.482127 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:53.482679 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:53.482690 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:53.484725 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
W20260810 22:39:53.485050 30335 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:53.479618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:60101 (local address 127.19.202.126:36639)
0810 22:39:53.479807 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:53.479810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:53.479900 (+ 90us) server_negotiation.cc:407] Connection header received
0810 22:39:53.479987 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:53.479989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:53.480032 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:53.480093 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:53.480488 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.481076 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.481763 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:53.481967 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:53.482037 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:53.484914 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:60101: BlockingRecv error: recv got EOF from 127.19.202.67:60101 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 10 22:39:54 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:54 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:54.141629 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:54.136066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:48999)
0810 22:39:54.136416 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:54.136430 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:54.136537 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:54.136877 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:54.136880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:54.137144 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:54.137265 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:54.137273 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.138062 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.138065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:54.138612 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:54.138620 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.139050 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.139053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:54.139117 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:54.139692 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:54.139704 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:54.141518 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":101,"threads_started":1}
W20260810 22:39:54.141800 30339 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:54.136145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:48999 (local address 127.19.202.126:36639)
0810 22:39:54.136423 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:54.136426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:54.136508 (+ 82us) server_negotiation.cc:407] Connection header received
0810 22:39:54.136634 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:54.136636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:54.136684 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:54.136798 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:54.137390 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.137957 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.138717 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.138946 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.139031 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:54.141664 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:48999: BlockingRecv error: recv got EOF from 127.19.202.65:48999 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":73,"threads_started":1}
Aug 10 22:39:54 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:54 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:54.322669 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:54.317477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:57507)
0810 22:39:54.317748 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:54.317761 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:54.317889 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:54.318155 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:54.318157 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:54.318361 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:54.318449 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:54.318454 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.319167 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.319172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:54.319718 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:54.319726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.320183 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.320187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:54.320272 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:54.320759 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:54.320770 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:54.322523 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":80,"threads_started":1}
W20260810 22:39:54.322733 30339 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:54.317549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:57507 (local address 127.19.202.126:36639)
0810 22:39:54.317711 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:54.317716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:54.317849 (+ 133us) server_negotiation.cc:407] Connection header received
0810 22:39:54.318001 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:54.318004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:54.318052 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:54.318131 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:54.318560 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.319047 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.319848 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.320071 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.320178 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:54.322633 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:57507: BlockingRecv error: recv got EOF from 127.19.202.66:57507 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 22:39:54 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:54 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:54.490962 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:54.485793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:53211)
0810 22:39:54.486059 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:54.486071 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:54.486168 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:54.486398 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:54.486400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:54.486581 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:54.486682 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:54.486687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.487367 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.487369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:54.487873 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:54.487880 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.488281 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.488283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:54.488337 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:54.488809 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:54.488820 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:54.490774 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
W20260810 22:39:54.491066 30339 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:54.485885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:53211 (local address 127.19.202.126:36639)
0810 22:39:54.486034 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:54.486038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:54.486140 (+ 102us) server_negotiation.cc:407] Connection header received
0810 22:39:54.486246 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:54.486248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:54.486293 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:54.486357 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:54.486798 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.487282 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.487984 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:54.488195 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:54.488262 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:54.490951 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:53211: BlockingRecv error: recv got EOF from 127.19.202.67:53211 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 22:39:55 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:55 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:55.148499 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:55.142545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:50189)
0810 22:39:55.142917 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:55.142935 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:55.143045 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:55.143348 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:55.143351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:55.143574 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:55.143674 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:55.143680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.144442 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.144446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:55.145121 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:55.145130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.145632 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.145635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:55.145725 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:55.146236 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:55.146248 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:55.148366 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":121,"threads_started":1}
W20260810 22:39:55.148653 30343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:55.142594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:50189 (local address 127.19.202.126:36639)
0810 22:39:55.142845 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:55.142848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:55.143020 (+ 172us) server_negotiation.cc:407] Connection header received
0810 22:39:55.143162 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:55.143165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:55.143210 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:55.143313 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:55.143796 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.144325 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.145240 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.145504 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.145620 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:55.148495 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:50189: BlockingRecv error: recv got EOF from 127.19.202.65:50189 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
Aug 10 22:39:55 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:55 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:55.328878 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:55.323460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:49639)
0810 22:39:55.323732 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:55.323745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:55.323839 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:55.324083 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:55.324085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:55.324260 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:55.324352 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:55.324357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.325048 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.325051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:55.325666 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:55.325720 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.326344 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.326347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:55.326450 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:55.327044 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:55.327055 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:55.328703 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
W20260810 22:39:55.328984 30343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:55.323575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:49639 (local address 127.19.202.126:36639)
0810 22:39:55.323718 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:55.323721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:55.323798 (+ 77us) server_negotiation.cc:407] Connection header received
0810 22:39:55.323933 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:55.323942 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:55.323982 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:55.324050 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:55.324449 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.324957 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.325841 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.326205 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.326320 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:55.328879 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:49639: BlockingRecv error: recv got EOF from 127.19.202.66:49639 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 22:39:55 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:55 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:55.496966 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:55.491694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:60347)
0810 22:39:55.492025 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:55.492038 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:55.492140 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:55.492357 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:55.492359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:55.492543 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:55.492641 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:55.492646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.493362 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.493365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:55.493882 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:55.493889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.494298 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.494300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:55.494360 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:55.494829 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:55.494840 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:55.496766 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":129,"threads_started":1}
W20260810 22:39:55.497033 30343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:55.491769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:60347 (local address 127.19.202.126:36639)
0810 22:39:55.491919 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:55.491924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:55.492111 (+ 187us) server_negotiation.cc:407] Connection header received
0810 22:39:55.492217 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:55.492219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:55.492264 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:55.492322 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:55.492746 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.493273 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.494000 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:55.494206 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:55.494278 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:55.496919 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:60347: BlockingRecv error: recv got EOF from 127.19.202.67:60347 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 22:39:56 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:56 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:56.155035 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:56.149337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:33245)
0810 22:39:56.149648 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:56.149661 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:56.149746 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:56.150034 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:56.150040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:56.150256 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:56.150372 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:56.150384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.151130 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.151133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:56.151681 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:56.151689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.152135 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.152137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:56.152199 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:56.152674 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:56.152685 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:56.154821 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":116,"threads_started":1}
W20260810 22:39:56.155182 30347 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:56.149419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:33245 (local address 127.19.202.126:36639)
0810 22:39:56.149729 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:56.149732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:56.149744 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:39:56.149834 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:56.149836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:56.149880 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:56.149950 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:56.150498 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.151031 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.151795 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.152036 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.152116 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:56.155032 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:33245: BlockingRecv error: recv got EOF from 127.19.202.65:33245 (error 108)
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":64,"threads_started":1}
Aug 10 22:39:56 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:56 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:56.339124 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:56.329646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:53709)
0810 22:39:56.329946 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:56.329959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:56.330129 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:56.330416 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:56.330419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:56.330601 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:56.330696 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:56.330701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.331382 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.331384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:56.331932 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:56.331940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.332320 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.332323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:56.332387 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:56.336971 (+ 4584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:56.336987 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:56.339008 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":130,"threads_started":1}
W20260810 22:39:56.339293 30347 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:56.329698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:53709 (local address 127.19.202.126:36639)
0810 22:39:56.329831 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:56.329835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:56.330063 (+ 228us) server_negotiation.cc:407] Connection header received
0810 22:39:56.330255 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:56.330257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:56.330302 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:56.330380 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:56.330802 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.331285 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.332036 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.332231 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.332299 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:56.339138 (+ 6839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:53709: BlockingRecv error: recv got EOF from 127.19.202.66:53709 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 22:39:56 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:56 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.67@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:56.503334 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:56.497793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.67:37149)
0810 22:39:56.498114 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:56.498127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:56.498256 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:56.498522 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:56.498525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:56.498755 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:56.498872 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:56.498879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.499631 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.499635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:56.500175 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:56.500182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.500666 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.500669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:56.500770 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:56.501348 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:56.501360 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:56.503210 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":133,"threads_started":1}
W20260810 22:39:56.503473 30347 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:56.497855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:37149 (local address 127.19.202.126:36639)
0810 22:39:56.498015 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:56.498019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:56.498206 (+ 187us) server_negotiation.cc:407] Connection header received
0810 22:39:56.498356 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:56.498358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:56.498407 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:56.498498 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:56.498992 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.499515 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.500288 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:56.500535 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:56.500649 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:56.503336 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.67:37149: BlockingRecv error: recv got EOF from 127.19.202.67:37149 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 22:39:57 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:57 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.65@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:57.161167 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:57.155876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.65:50913)
0810 22:39:57.156159 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:57.156173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:57.156257 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:57.156513 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:57.156515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:57.156712 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:57.156830 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:57.156835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.157583 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.157585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:57.158121 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:57.158128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.158560 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.158563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:57.158626 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:57.159187 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:57.159200 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:57.161045 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
W20260810 22:39:57.161331 30351 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:57.155958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:50913 (local address 127.19.202.126:36639)
0810 22:39:57.156225 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:57.156229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:57.156244 (+ 15us) server_negotiation.cc:407] Connection header received
0810 22:39:57.156349 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:57.156352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:57.156396 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:57.156483 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:57.156940 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.157480 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.158215 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.158458 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.158545 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:57.161188 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.65:50913: BlockingRecv error: recv got EOF from 127.19.202.65:50913 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":104,"threads_started":1}
Aug 10 22:39:57 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:39:57 dist-test-slave-1x4g krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.202.66@KRBTEST.COM for oryx/127.19.202.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:39:57.345201 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:57.339918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:36639 (local address 127.19.202.66:36103)
0810 22:39:57.340234 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:57.340248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:57.340378 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:57.340659 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:57.340674 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:57.340918 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:57.341011 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:57.341016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.341726 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.341731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:57.342271 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:57.342279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.342717 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.342720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:57.342802 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:57.343252 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:39:57.343263 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:57.345090 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.202.126:36639: Server oryx/127.19.202.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
W20260810 22:39:57.345330 30351 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:57.340032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:36103 (local address 127.19.202.126:36639)
0810 22:39:57.340195 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:57.340200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:57.340332 (+ 132us) server_negotiation.cc:407] Connection header received
0810 22:39:57.340486 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:57.340488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:57.340542 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:57.340630 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:57.341123 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.341619 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.342391 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.342611 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.342712 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:57.345219 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.202.66:36103: BlockingRecv error: recv got EOF from 127.19.202.66:36103 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:39:57.403124 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 29841
I20260810 22:39:57.411971 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30005
I20260810 22:39:57.418639 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30146
I20260810 22:39:57.425559 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 29765
2026-08-10T22:39:57Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17942 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T22:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:57Z Disabled control of system clock
I20260810 22:39:57.457885 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38411
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:38411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 22:39:57.565413 30356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:57.565641 30356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:57.565673 30356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:57.569413 30356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:57.569464 30356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:57.569480 30356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:57.569496 30356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:57.569511 30356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:57.573881 30356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:38411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:57.574824 30356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:57.576104 30356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:39:57.584654 30356 server_base.cc:1034] running on GCE node
W20260810 22:39:57.584630 30361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:57.584649 30364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:57.584685 30362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:57.585533 30356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:57.586570 30356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:57.587757 30356 hybrid_clock.cc:648] HybridClock initialized: now 1786401597587739 us; error 36 us; skew 500 ppm
I20260810 22:39:57.589564 30356 webserver.cc:460] Webserver started at http://127.19.202.126:36735/ using document root <none> and password file <none>
I20260810 22:39:57.590126 30356 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:57.590216 30356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:57.590462 30356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:57.592355 30356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "c7955544ca514ac88a770b08ce35334c"
format_stamp: "Formatted at 2026-08-10 22:39:57 on dist-test-slave-1x4g"
server_key: "4151e915d515dc53b6b5bbf5ebfb6639"
server_key_iv: "96ec55a417a0f8e20e75044f7ec32e62"
server_key_version: "encryptionkey@0"
I20260810 22:39:57.592937 30356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "c7955544ca514ac88a770b08ce35334c"
format_stamp: "Formatted at 2026-08-10 22:39:57 on dist-test-slave-1x4g"
server_key: "4151e915d515dc53b6b5bbf5ebfb6639"
server_key_iv: "96ec55a417a0f8e20e75044f7ec32e62"
server_key_version: "encryptionkey@0"
I20260810 22:39:57.596402 30356 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:57.598881 30370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:57.599988 30356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:57.600118 30356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "c7955544ca514ac88a770b08ce35334c"
format_stamp: "Formatted at 2026-08-10 22:39:57 on dist-test-slave-1x4g"
server_key: "4151e915d515dc53b6b5bbf5ebfb6639"
server_key_iv: "96ec55a417a0f8e20e75044f7ec32e62"
server_key_version: "encryptionkey@0"
I20260810 22:39:57.600221 30356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:57.629781 30356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:57.630546 30356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:57.630748 30356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:57.637701 30356 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:38411
I20260810 22:39:57.637722 30426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:38411 every 8 connection(s)
I20260810 22:39:57.638604 30356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:57.641758 30427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:57.644054 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30356
I20260810 22:39:57.644150 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:57.644438 20265 external_mini_cluster.cc:1428] Setting key 6b7bc33fff3ff6799c9f91dfc1d14c13
I20260810 22:39:57.647786 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c: Bootstrap starting.
I20260810 22:39:57.650449 30427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:57.651363 30427 log.cc:824] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:57.653538 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c: No bootstrap required, opened a new log
I20260810 22:39:57.653805 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:57.645941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.19.202.126:38411)
0810 22:39:57.646525 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:57.646541 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:57.646575 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:39:57.647191 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:57.647205 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:57.647580 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:57.647827 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:57.648234 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.649248 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.650467 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.650958 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.651080 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:57.651995 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:57.652003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:57.652020 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:57.652423 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:57.652474 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:57.652494 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:57.652563 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:57.652734 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 22:39:57.653069 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":125,"threads_started":1}
I20260810 22:39:57.657130 30427 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7955544ca514ac88a770b08ce35334c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38411 } }
I20260810 22:39:57.657352 30427 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:57.657445 30427 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7955544ca514ac88a770b08ce35334c, State: Initialized, Role: FOLLOWER
I20260810 22:39:57.657840 30427 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7955544ca514ac88a770b08ce35334c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38411 } }
I20260810 22:39:57.657960 30427 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:57.658058 30427 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:57.658176 30427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:57.659085 30427 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7955544ca514ac88a770b08ce35334c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38411 } }
I20260810 22:39:57.659404 30427 leader_election.cc:302] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7955544ca514ac88a770b08ce35334c; no voters:
I20260810 22:39:57.659698 30427 leader_election.cc:288] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:57.659788 30433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:57.660054 30433 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [term 1 LEADER]: Becoming Leader. State: Replica: c7955544ca514ac88a770b08ce35334c, State: Running, Role: LEADER
I20260810 22:39:57.660347 30433 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7955544ca514ac88a770b08ce35334c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38411 } }
I20260810 22:39:57.660965 30427 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:57.662045 30434 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7955544ca514ac88a770b08ce35334c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7955544ca514ac88a770b08ce35334c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38411 } } }
I20260810 22:39:57.662032 30435 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7955544ca514ac88a770b08ce35334c. Latest consensus state: current_term: 1 leader_uuid: "c7955544ca514ac88a770b08ce35334c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7955544ca514ac88a770b08ce35334c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38411 } } }
I20260810 22:39:57.662171 30434 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:57.662171 30435 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:57.662572 30442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:57.665321 30442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:57.670486 30442 catalog_manager.cc:1347] Generated new cluster ID: 79abcc4bbe684defba4b60ba25496f78
I20260810 22:39:57.670542 30442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:57.681239 30442 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:57.682111 30442 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:57.686046 30442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7955544ca514ac88a770b08ce35334c: Generated new TSK 0
I20260810 22:39:57.686688 30442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:39:57.689548 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38411
--builtin_ntp_servers=127.19.202.84:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 22:39:57.801180 30455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:57.801407 30455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:57.801450 30455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:57.805543 30455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:57.805590 30455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:57.805660 30455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:57.810365 30455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:57.811401 30455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:57.812875 30455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:57.822072 30463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:57.822065 30461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:57.822074 30460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:57.822121 30455 server_base.cc:1034] running on GCE node
I20260810 22:39:57.822687 30455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:57.823232 30455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:57.824386 30455 hybrid_clock.cc:648] HybridClock initialized: now 1786401597824372 us; error 25 us; skew 500 ppm
I20260810 22:39:57.826269 30455 webserver.cc:460] Webserver started at http://127.19.202.65:42873/ using document root <none> and password file <none>
I20260810 22:39:57.826857 30455 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:57.826963 30455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:57.827198 30455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:57.829226 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "f9afa4b71b3744b184ff66f26332a854"
format_stamp: "Formatted at 2026-08-10 22:39:57 on dist-test-slave-1x4g"
server_key: "10cc613ad0f74d122f310c9a2b72f02d"
server_key_iv: "b81a2d724d2d65a18266fa90f7506193"
server_key_version: "encryptionkey@0"
I20260810 22:39:57.829777 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "f9afa4b71b3744b184ff66f26332a854"
format_stamp: "Formatted at 2026-08-10 22:39:57 on dist-test-slave-1x4g"
server_key: "10cc613ad0f74d122f310c9a2b72f02d"
server_key_iv: "b81a2d724d2d65a18266fa90f7506193"
server_key_version: "encryptionkey@0"
I20260810 22:39:57.833739 30455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 22:39:57.837256 30469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:57.838989 30455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:39:57.839110 30455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "f9afa4b71b3744b184ff66f26332a854"
format_stamp: "Formatted at 2026-08-10 22:39:57 on dist-test-slave-1x4g"
server_key: "10cc613ad0f74d122f310c9a2b72f02d"
server_key_iv: "b81a2d724d2d65a18266fa90f7506193"
server_key_version: "encryptionkey@0"
I20260810 22:39:57.839241 30455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:57.854830 30455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:57.855525 30455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:57.855731 30455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:57.856320 30455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:57.857344 30455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:57.857425 30455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:57.857517 30455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:57.857568 30455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:57.866369 30455 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:43887
I20260810 22:39:57.866449 30590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:43887 every 8 connection(s)
I20260810 22:39:57.867265 30455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:57.876595 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30455
I20260810 22:39:57.876726 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:57.876734 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:57.869072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:42685 (local address 127.19.202.126:38411)
0810 22:39:57.869211 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:57.869221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:57.870199 (+ 978us) server_negotiation.cc:407] Connection header received
0810 22:39:57.871688 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:57.871692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:57.871776 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:57.871896 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:57.872832 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.873391 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.874603 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.874861 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.874980 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:57.876145 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:57.876150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:57.876152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:57.876167 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:57.876198 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:57.876208 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:57.876297 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:57.876495 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 22:39:57.876561 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:39:57.877076 20265 external_mini_cluster.cc:1428] Setting key 3ae64b10fadd6738051b26b00158da07
I20260810 22:39:57.877671 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:57.869397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38411 (local address 127.19.202.65:42685)
0810 22:39:57.870046 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:57.870086 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:57.871426 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:57.872070 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:57.872077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:57.872168 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:57.872665 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:57.872679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.873540 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.873544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:57.874464 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:57.874474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:57.875021 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:57.875026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:57.875198 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:57.875854 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:57.875855 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:57.875913 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:57.875978 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:57.876300 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:57.876313 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:57.876321 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 22:39:57.876478 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 22:39:57.876767 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
I20260810 22:39:57.878914 30591 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38411
I20260810 22:39:57.879120 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38411
--builtin_ntp_servers=127.19.202.84:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 22:39:57.879171 30591 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:57.879654 30591 heartbeater.cc:511] Master 127.19.202.126:38411 requested a full tablet report, sending...
I20260810 22:39:57.881287 30387 ts_manager.cc:194] Registered new tserver with Master: f9afa4b71b3744b184ff66f26332a854 (127.19.202.65:43887)
I20260810 22:39:57.882711 30387 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.65:42685
W20260810 22:39:57.986327 30594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:57.986552 30594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:57.986585 30594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:57.990237 30594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:57.990278 30594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:57.990348 30594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:57.995246 30594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:57.996188 30594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:57.997589 30594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:58.007062 30602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:58.007118 30600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.007205 30594 server_base.cc:1034] running on GCE node
W20260810 22:39:58.007114 30599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.007664 30594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:58.008246 30594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:58.009423 30594 hybrid_clock.cc:648] HybridClock initialized: now 1786401598009395 us; error 46 us; skew 500 ppm
I20260810 22:39:58.011206 30594 webserver.cc:460] Webserver started at http://127.19.202.66:46815/ using document root <none> and password file <none>
I20260810 22:39:58.011767 30594 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:58.011857 30594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:58.012077 30594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:58.014077 30594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "3afed3cf9ebf430ebfceca7164888ba6"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "fc9a02c26b1a25551a60bbbd15475c9b"
server_key_iv: "c77b583fa8a2a5109fbb36eab8e57fbd"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.014616 30594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "3afed3cf9ebf430ebfceca7164888ba6"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "fc9a02c26b1a25551a60bbbd15475c9b"
server_key_iv: "c77b583fa8a2a5109fbb36eab8e57fbd"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.018112 30594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 22:39:58.020228 30608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.021422 30594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:58.021553 30594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "3afed3cf9ebf430ebfceca7164888ba6"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "fc9a02c26b1a25551a60bbbd15475c9b"
server_key_iv: "c77b583fa8a2a5109fbb36eab8e57fbd"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.021657 30594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:58.047619 30594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:58.048197 30594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:58.048377 30594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:58.048945 30594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:58.049815 30594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:58.049888 30594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.049954 30594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:58.050000 30594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.059158 30594 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:39577
I20260810 22:39:58.059197 30736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:39577 every 8 connection(s)
I20260810 22:39:58.059971 30594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:58.066407 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30594
I20260810 22:39:58.066545 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:58.066856 20265 external_mini_cluster.cc:1428] Setting key d6b028e841300f7f304a91973f6d76b1
I20260810 22:39:58.068827 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.061805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:47869 (local address 127.19.202.126:38411)
0810 22:39:58.062001 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:58.062005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:58.062940 (+ 935us) server_negotiation.cc:407] Connection header received
0810 22:39:58.064237 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:58.064240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:58.064283 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:58.064352 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:58.065193 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.065690 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.066909 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.067197 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.067305 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:58.068335 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:58.068339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:58.068341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:58.068351 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:58.068385 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:58.068393 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:58.068465 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:58.068620 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 22:39:58.068675 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260810 22:39:58.069048 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38411
--builtin_ntp_servers=127.19.202.84:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 22:39:58.069648 30739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.062178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38411 (local address 127.19.202.66:47869)
0810 22:39:58.062818 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:58.062854 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:58.064044 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:58.064461 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:58.064467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:58.064534 (+ 67us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:58.065066 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:58.065079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.065799 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.065801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:58.066769 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:58.066779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.067339 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.067343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:58.067499 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:58.068075 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:58.068077 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:58.068122 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:58.068194 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:58.068467 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:58.068478 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:58.068485 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 22:39:58.068609 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 22:39:58.068872 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":111,"threads_started":1}
I20260810 22:39:58.070686 30737 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38411
I20260810 22:39:58.070942 30737 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:58.071463 30737 heartbeater.cc:511] Master 127.19.202.126:38411 requested a full tablet report, sending...
I20260810 22:39:58.072536 30387 ts_manager.cc:194] Registered new tserver with Master: 3afed3cf9ebf430ebfceca7164888ba6 (127.19.202.66:39577)
I20260810 22:39:58.073169 30387 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.66:47869
W20260810 22:39:58.176849 30740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:58.177099 30740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:58.177170 30740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:58.180958 30740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:58.181023 30740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:58.181095 30740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:58.185485 30740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:58.186468 30740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:58.187737 30740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:58.196817 30746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:58.196947 30745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.196969 30740 server_base.cc:1034] running on GCE node
W20260810 22:39:58.196816 30748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.197485 30740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:58.198123 30740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:58.199304 30740 hybrid_clock.cc:648] HybridClock initialized: now 1786401598199288 us; error 29 us; skew 500 ppm
I20260810 22:39:58.201057 30740 webserver.cc:460] Webserver started at http://127.19.202.67:40357/ using document root <none> and password file <none>
I20260810 22:39:58.201597 30740 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:58.201684 30740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:58.201905 30740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:58.203996 30740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "0fc9866eeed3474d957ac00c1a47b21f"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "b7f32c8d4c020fd8dce6b8461b751077"
server_key_iv: "0b5026a3d9a65d4db8dec8d036bf3042"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.204568 30740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "0fc9866eeed3474d957ac00c1a47b21f"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "b7f32c8d4c020fd8dce6b8461b751077"
server_key_iv: "0b5026a3d9a65d4db8dec8d036bf3042"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.208127 30740 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:39:58.210603 30754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.211826 30740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:58.211962 30740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "0fc9866eeed3474d957ac00c1a47b21f"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "b7f32c8d4c020fd8dce6b8461b751077"
server_key_iv: "0b5026a3d9a65d4db8dec8d036bf3042"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.212070 30740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:58.222780 30740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:58.223393 30740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:58.223582 30740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:58.224154 30740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:58.225184 30740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:58.225250 30740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.225312 30740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:58.225358 30740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.233769 30740 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:33719
I20260810 22:39:58.233793 30874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:33719 every 8 connection(s)
I20260810 22:39:58.234560 30740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:58.236694 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30740
I20260810 22:39:58.236824 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:58.237126 20265 external_mini_cluster.cc:1428] Setting key 9dd906a7662825f2f6cc926c315f3a5d
I20260810 22:39:58.243952 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.236353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:55563 (local address 127.19.202.126:38411)
0810 22:39:58.236576 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:58.236583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:58.237759 (+ 1176us) server_negotiation.cc:407] Connection header received
0810 22:39:58.239348 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:58.239351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:58.239402 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:58.239471 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:58.240335 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.240937 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.242110 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.242347 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.242452 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:58.243456 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:58.243460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:58.243462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:58.243477 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:58.243499 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:58.243514 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:58.243589 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:58.243746 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 22:39:58.243807 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 22:39:58.244874 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.236766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38411 (local address 127.19.202.67:55563)
0810 22:39:58.237590 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:58.237641 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:58.239139 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:58.239640 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:58.239647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:58.239728 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:58.240189 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:58.240200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.241069 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.241072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:58.241985 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:39:58.241995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:39:58.242495 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:39:58.242499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:39:58.242660 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:39:58.243186 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:58.243188 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:58.243225 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:58.243284 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:58.243593 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:58.243604 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:58.243610 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 22:39:58.243746 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 22:39:58.244008 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":147,"threads_started":1}
I20260810 22:39:58.245885 30875 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38411
I20260810 22:39:58.246176 30875 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:58.246708 30875 heartbeater.cc:511] Master 127.19.202.126:38411 requested a full tablet report, sending...
I20260810 22:39:58.247720 30387 ts_manager.cc:194] Registered new tserver with Master: 0fc9866eeed3474d957ac00c1a47b21f (127.19.202.67:33719)
I20260810 22:39:58.248266 30387 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.67:55563
I20260810 22:39:58.250943 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 22:39:58.253548 30431 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:58.252771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53292 (local address 127.19.202.126:38411)
0810 22:39:58.252899 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:58.252902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:58.253088 (+ 186us) server_negotiation.cc:407] Connection header received
0810 22:39:58.253173 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:58.253175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:58.253208 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:58.253285 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:58.253463 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53292: BlockingRecv error: recv got EOF from 127.0.0.1:53292 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 22:39:58.253767 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30455
I20260810 22:39:58.266892 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30594
I20260810 22:39:58.274147 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30740
I20260810 22:39:58.280623 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30356
2026-08-10T22:39:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (854 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T22:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:39:58Z Disabled control of system clock
I20260810 22:39:58.311661 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:43325
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:43325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 22:39:58.421407 30887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:58.421640 30887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:58.421674 30887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:58.425339 30887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:39:58.425382 30887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:58.425398 30887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:58.425415 30887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:39:58.425429 30887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:39:58.429955 30887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:43325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:43325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:58.430907 30887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:58.432161 30887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:58.440439 30892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:58.440439 30895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.440523 30887 server_base.cc:1034] running on GCE node
W20260810 22:39:58.440511 30893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.441270 30887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:58.442312 30887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:58.443509 30887 hybrid_clock.cc:648] HybridClock initialized: now 1786401598443502 us; error 40 us; skew 500 ppm
I20260810 22:39:58.445300 30887 webserver.cc:460] Webserver started at http://127.19.202.126:41637/ using document root <none> and password file <none>
I20260810 22:39:58.445858 30887 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:58.445977 30887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:58.446213 30887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:58.448210 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "081e40a07601406cafd9d40d606c8aff"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "c4cff7c80b78ac0b195a59cd39624568"
server_key_iv: "6698279720f4038a295994f429e170c4"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.448702 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "081e40a07601406cafd9d40d606c8aff"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "c4cff7c80b78ac0b195a59cd39624568"
server_key_iv: "6698279720f4038a295994f429e170c4"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.452091 30887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:58.454334 30902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.455590 30887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:39:58.455732 30887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "081e40a07601406cafd9d40d606c8aff"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "c4cff7c80b78ac0b195a59cd39624568"
server_key_iv: "6698279720f4038a295994f429e170c4"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.455835 30887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:58.460320 30887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:58.460969 30887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:58.461149 30887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:58.467979 30887 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:43325
I20260810 22:39:58.467998 30961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:43325 every 8 connection(s)
I20260810 22:39:58.468966 30887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:39:58.472214 30962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:39:58.477010 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30887
I20260810 22:39:58.477165 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:39:58.477443 20265 external_mini_cluster.cc:1428] Setting key eee5dde221528621337073e713486f42
I20260810 22:39:58.477847 30962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff: Bootstrap starting.
I20260810 22:39:58.480397 30962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff: Neither blocks nor log segments found. Creating new log.
I20260810 22:39:58.481276 30962 log.cc:824] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff: Log is configured to *not* fsync() on all Append() calls
I20260810 22:39:58.483536 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.478830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.19.202.126:43325)
0810 22:39:58.479535 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:58.479554 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:58.479595 (+ 41us) server_negotiation.cc:407] Connection header received
0810 22:39:58.480479 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:58.480504 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:58.480967 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:58.481309 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:58.482139 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:58.482144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:58.482155 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:58.482165 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:58.482247 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:58.482272 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:58.482381 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:58.482529 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 22:39:58.482760 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":118,"threads_started":1}
I20260810 22:39:58.483561 30962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff: No bootstrap required, opened a new log
I20260810 22:39:58.487278 30962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081e40a07601406cafd9d40d606c8aff" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43325 } }
I20260810 22:39:58.487485 30962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:39:58.487519 30962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 081e40a07601406cafd9d40d606c8aff, State: Initialized, Role: FOLLOWER
I20260810 22:39:58.487859 30962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081e40a07601406cafd9d40d606c8aff" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43325 } }
I20260810 22:39:58.487967 30962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:39:58.488008 30962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:39:58.488071 30962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:39:58.488860 30962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081e40a07601406cafd9d40d606c8aff" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43325 } }
I20260810 22:39:58.489154 30962 leader_election.cc:302] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 081e40a07601406cafd9d40d606c8aff; no voters:
I20260810 22:39:58.489391 30962 leader_election.cc:288] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:39:58.489521 30967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:39:58.489759 30967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [term 1 LEADER]: Becoming Leader. State: Replica: 081e40a07601406cafd9d40d606c8aff, State: Running, Role: LEADER
I20260810 22:39:58.490059 30967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081e40a07601406cafd9d40d606c8aff" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43325 } }
I20260810 22:39:58.490593 30962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:39:58.491757 30969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 081e40a07601406cafd9d40d606c8aff. Latest consensus state: current_term: 1 leader_uuid: "081e40a07601406cafd9d40d606c8aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081e40a07601406cafd9d40d606c8aff" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43325 } } }
I20260810 22:39:58.491884 30969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:58.492269 30977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:39:58.492431 30968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "081e40a07601406cafd9d40d606c8aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081e40a07601406cafd9d40d606c8aff" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 43325 } } }
I20260810 22:39:58.492569 30968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff [sys.catalog]: This master's current role is: LEADER
I20260810 22:39:58.495390 30977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:39:58.500663 30977 catalog_manager.cc:1347] Generated new cluster ID: acaaf4a743004fb2bb136894a6fb2224
I20260810 22:39:58.500726 30977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:39:58.514863 30977 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:39:58.515189 30977 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:39:58.519641 30977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 081e40a07601406cafd9d40d606c8aff: Generated new TSK 0
I20260810 22:39:58.520244 30977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:39:58.524945 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43325
--builtin_ntp_servers=127.19.202.84:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 22:39:58.636699 30988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:58.636958 30988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:58.637030 30988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:58.640913 30988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:58.640959 30988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:58.641052 30988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:39:58.645390 30988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:58.646391 30988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:58.647945 30988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:58.657166 30995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:58.657166 30993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.657310 30988 server_base.cc:1034] running on GCE node
W20260810 22:39:58.657235 30997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.657967 30988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:58.658650 30988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:58.659802 30988 hybrid_clock.cc:648] HybridClock initialized: now 1786401598659781 us; error 34 us; skew 500 ppm
I20260810 22:39:58.661533 30988 webserver.cc:460] Webserver started at http://127.19.202.65:44865/ using document root <none> and password file <none>
I20260810 22:39:58.662137 30988 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:58.662195 30988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:58.662444 30988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:58.664427 30988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "56ad676e1504465ebcd0347911178a62"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "55796d43eae2759e90b44ff8280622b3"
server_key_iv: "d429c616a1263870e1a58f045c4715e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.665020 30988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "56ad676e1504465ebcd0347911178a62"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "55796d43eae2759e90b44ff8280622b3"
server_key_iv: "d429c616a1263870e1a58f045c4715e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.668437 30988 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:39:58.670837 31004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.671964 30988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:58.672086 30988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "56ad676e1504465ebcd0347911178a62"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "55796d43eae2759e90b44ff8280622b3"
server_key_iv: "d429c616a1263870e1a58f045c4715e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.672190 30988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:58.676534 30988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:58.677205 30988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:58.677386 30988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:58.677906 30988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:58.678804 30988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:58.678874 30988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.678941 30988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:58.678992 30988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.688020 30988 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:37299
I20260810 22:39:58.688059 31129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:37299 every 8 connection(s)
I20260810 22:39:58.688956 30988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:39:58.692130 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 30988
I20260810 22:39:58.692269 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:39:58.692543 20265 external_mini_cluster.cc:1428] Setting key 7f534769c0c85fb4ba9e65d2022c0899
I20260810 22:39:58.694507 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43325
--builtin_ntp_servers=127.19.202.84:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 22:39:58.695446 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.690716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:39545 (local address 127.19.202.126:43325)
0810 22:39:58.690894 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:58.690898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:58.692048 (+ 1150us) server_negotiation.cc:407] Connection header received
0810 22:39:58.693701 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:58.693704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:58.693745 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:58.693822 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:58.694969 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:58.694972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:58.694975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:58.694986 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:58.695009 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:58.695017 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:58.695109 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:58.695260 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 22:39:58.695304 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 22:39:58.696421 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.691150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:43325 (local address 127.19.202.65:39545)
0810 22:39:58.691801 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:58.691942 (+ 141us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:58.693500 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:58.693994 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:58.694001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:58.694081 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:58.694675 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:58.694679 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:58.694734 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:58.694819 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:58.695098 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:58.695108 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:58.695115 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 22:39:58.695299 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 22:39:58.695529 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":102,"threads_started":1}
I20260810 22:39:58.697650 31130 heartbeater.cc:347] Connected to a master server at 127.19.202.126:43325
I20260810 22:39:58.697886 31130 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:58.698321 31130 heartbeater.cc:511] Master 127.19.202.126:43325 requested a full tablet report, sending...
I20260810 22:39:58.699852 30920 ts_manager.cc:194] Registered new tserver with Master: 56ad676e1504465ebcd0347911178a62 (127.19.202.65:37299)
W20260810 22:39:58.804837 31133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:58.805080 31133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:58.805160 31133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:58.809007 31133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:58.809078 31133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:58.809166 31133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:39:58.813732 31133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:58.814708 31133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:58.816007 31133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:58.825229 31139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.825312 31133 server_base.cc:1034] running on GCE node
W20260810 22:39:58.825193 31138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:58.825218 31141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.825884 31133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:58.826498 31133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:58.827703 31133 hybrid_clock.cc:648] HybridClock initialized: now 1786401598827674 us; error 43 us; skew 500 ppm
I20260810 22:39:58.829553 31133 webserver.cc:460] Webserver started at http://127.19.202.66:34921/ using document root <none> and password file <none>
I20260810 22:39:58.830149 31133 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:58.830262 31133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:58.830497 31133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:58.832970 31133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "17927f3197394204addac443cc0611cc"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "956b7b27eb9dc928fab7753d60a3bbf1"
server_key_iv: "36ed0a2548d1b0cf355261a76dacd9e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.833675 31133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "17927f3197394204addac443cc0611cc"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "956b7b27eb9dc928fab7753d60a3bbf1"
server_key_iv: "36ed0a2548d1b0cf355261a76dacd9e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.838359 31133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 22:39:58.840821 31149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.842085 31133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:39:58.842218 31133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "17927f3197394204addac443cc0611cc"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "956b7b27eb9dc928fab7753d60a3bbf1"
server_key_iv: "36ed0a2548d1b0cf355261a76dacd9e9"
server_key_version: "encryptionkey@0"
I20260810 22:39:58.842329 31133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:58.846822 31133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:58.847415 31133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:58.847599 31133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:58.848117 31133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:58.849030 31133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:58.849108 31133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.849192 31133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:58.849241 31133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:58.858392 31133 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:38563
I20260810 22:39:58.858410 31275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:38563 every 8 connection(s)
I20260810 22:39:58.859314 31133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:39:58.862030 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 31133
I20260810 22:39:58.862188 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:39:58.862514 20265 external_mini_cluster.cc:1428] Setting key bf41510dc1b7e302d09d5f174a8991db
I20260810 22:39:58.864362 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43325
--builtin_ntp_servers=127.19.202.84:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 22:39:58.865861 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.861139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:42687 (local address 127.19.202.126:43325)
0810 22:39:58.861322 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:58.861327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:58.862490 (+ 1163us) server_negotiation.cc:407] Connection header received
0810 22:39:58.863895 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:58.863898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:58.863942 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:58.864058 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:58.865310 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:58.865323 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:58.865326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:58.865341 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:58.865374 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:58.865384 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:58.865490 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:58.865667 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 22:39:58.865715 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 22:39:58.866751 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:58.861520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:43325 (local address 127.19.202.66:42687)
0810 22:39:58.862317 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:58.862363 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:58.863674 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:58.864224 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:58.864231 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:58.864314 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:58.865044 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:58.865046 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:58.865086 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:58.865155 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:58.865477 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:58.865486 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:58.865494 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 22:39:58.865696 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 22:39:58.865958 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":109,"threads_started":1}
I20260810 22:39:58.867969 31276 heartbeater.cc:347] Connected to a master server at 127.19.202.126:43325
I20260810 22:39:58.868198 31276 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:58.868635 31276 heartbeater.cc:511] Master 127.19.202.126:43325 requested a full tablet report, sending...
I20260810 22:39:58.869880 30920 ts_manager.cc:194] Registered new tserver with Master: 17927f3197394204addac443cc0611cc (127.19.202.66:38563)
W20260810 22:39:58.975092 31280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:39:58.975322 31280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:39:58.975389 31280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:39:58.979346 31280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:39:58.979393 31280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:39:58.979496 31280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:39:58.984100 31280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:43325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:39:58.985162 31280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:39:58.986534 31280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:39:58.996263 31286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:58.996263 31287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:39:58.996351 31289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:39:58.996677 31280 server_base.cc:1034] running on GCE node
I20260810 22:39:58.997155 31280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:39:58.997748 31280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:39:58.998922 31280 hybrid_clock.cc:648] HybridClock initialized: now 1786401598998905 us; error 36 us; skew 500 ppm
I20260810 22:39:59.001003 31280 webserver.cc:460] Webserver started at http://127.19.202.67:45843/ using document root <none> and password file <none>
I20260810 22:39:59.001618 31280 fs_manager.cc:359] Metadata directory not provided
I20260810 22:39:59.001684 31280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:39:59.001866 31280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:39:59.003999 31280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "1b73241b4e5d4c4083bc7f767af3efd2"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "3863cbe56b022f6f6f64b54272345357"
server_key_iv: "be41e5e0c752e3cf36bfb3e614cc00e0"
server_key_version: "encryptionkey@0"
I20260810 22:39:59.004554 31280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "1b73241b4e5d4c4083bc7f767af3efd2"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "3863cbe56b022f6f6f64b54272345357"
server_key_iv: "be41e5e0c752e3cf36bfb3e614cc00e0"
server_key_version: "encryptionkey@0"
I20260810 22:39:59.008309 31280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:39:59.010919 31295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:59.012112 31280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:39:59.012209 31280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "1b73241b4e5d4c4083bc7f767af3efd2"
format_stamp: "Formatted at 2026-08-10 22:39:58 on dist-test-slave-1x4g"
server_key: "3863cbe56b022f6f6f64b54272345357"
server_key_iv: "be41e5e0c752e3cf36bfb3e614cc00e0"
server_key_version: "encryptionkey@0"
I20260810 22:39:59.012293 31280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:39:59.016660 31280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:39:59.017285 31280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:39:59.017423 31280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:39:59.017903 31280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:39:59.018695 31280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:39:59.018738 31280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:59.018771 31280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:39:59.018790 31280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:39:59.027426 31280 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:46021
I20260810 22:39:59.027449 31419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:46021 every 8 connection(s)
I20260810 22:39:59.028270 31280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:39:59.031611 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 31280
I20260810 22:39:59.031716 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:39:59.032006 20265 external_mini_cluster.cc:1428] Setting key 1249e1cf41280545454e9f68581e797d
I20260810 22:39:59.034749 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:59.030000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:44957 (local address 127.19.202.126:43325)
0810 22:39:59.030151 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:59.030156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:59.031237 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 22:39:59.032954 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:59.032957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:59.033011 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:59.033110 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:59.034333 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:39:59.034336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:39:59.034339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:39:59.034355 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:39:59.034380 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:39:59.034390 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:39:59.034475 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 22:39:59.034616 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 22:39:59.034652 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:39:59.035553 31422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:39:59.030453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:43325 (local address 127.19.202.67:44957)
0810 22:39:59.031099 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 22:39:59.031136 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 22:39:59.032719 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:39:59.033273 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:39:59.033281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:39:59.033393 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:39:59.034047 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:39:59.034049 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:39:59.034087 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:39:59.034184 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:39:59.034489 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:39:59.034496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:39:59.034501 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:39:59.034631 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 22:39:59.034805 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260810 22:39:59.036536 31420 heartbeater.cc:347] Connected to a master server at 127.19.202.126:43325
I20260810 22:39:59.036768 31420 heartbeater.cc:464] Registering TS with master...
I20260810 22:39:59.037292 31420 heartbeater.cc:511] Master 127.19.202.126:43325 requested a full tablet report, sending...
I20260810 22:39:59.038293 30920 ts_manager.cc:194] Registered new tserver with Master: 1b73241b4e5d4c4083bc7f767af3efd2 (127.19.202.67:46021)
I20260810 22:39:59.045904 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 22:39:59.049218 30965 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:39:59.048213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.19.202.126:43325)
0810 22:39:59.048391 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 22:39:59.048395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:39:59.048551 (+ 156us) server_negotiation.cc:407] Connection header received
0810 22:39:59.048643 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:39:59.048645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:39:59.048674 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:39:59.048739 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:39:59.049084 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37208: BlockingRecv error: recv got EOF from 127.0.0.1:37208 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:39:59.049268 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30988
I20260810 22:39:59.060322 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 31133
I20260810 22:39:59.072669 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 31280
I20260810 22:39:59.078585 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 30887
2026-08-10T22:39:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (797 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:39:59 dist-test-slave-1x4g krb5kdc[31432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:39:59 dist-test-slave-1x4g krb5kdc[31432](info): set up 2 sockets
Aug 10 22:39:59 dist-test-slave-1x4g krb5kdc[31432](info): commencing operation
krb5kdc: starting...
W20260810 22:40:00.127820 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:40:00 dist-test-slave-1x4g krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:40:00.251606 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34751
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:40:00.363066 31446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:00.363339 31446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:00.363431 31446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:00.367441 31446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:40:00.367519 31446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:00.367551 31446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:00.367585 31446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:40:00.367612 31446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:40:00.372711 31446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:00.373713 31446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:00.374677 31446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:00.383152 31454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:00.383193 31446 server_base.cc:1034] running on GCE node
W20260810 22:40:00.383152 31452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:00.383142 31451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:00.383991 31446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:00.385110 31446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:00.386327 31446 hybrid_clock.cc:648] HybridClock initialized: now 1786401600386304 us; error 40 us; skew 500 ppm
Aug 10 22:40:00 dist-test-slave-1x4g krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401600, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:00.389263 31446 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:40:00.390172 31446 webserver.cc:460] Webserver started at http://127.19.202.126:37369/ using document root <none> and password file <none>
I20260810 22:40:00.390672 31446 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:00.390728 31446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:00.390918 31446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:40:00.393013 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "6a5bc8819cab446fa6e874d626f083b8"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "af97b340b276ff1b150e5864b0b39edb"
server_key_iv: "58f90aef374342838f1df6ee25585ed1"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.393525 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "6a5bc8819cab446fa6e874d626f083b8"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "af97b340b276ff1b150e5864b0b39edb"
server_key_iv: "58f90aef374342838f1df6ee25585ed1"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.397228 31446 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:40:00.399462 31462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:00.400660 31446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:40:00.400785 31446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "6a5bc8819cab446fa6e874d626f083b8"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "af97b340b276ff1b150e5864b0b39edb"
server_key_iv: "58f90aef374342838f1df6ee25585ed1"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.400925 31446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:40:00.415344 31446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:00.418605 31446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:00.418823 31446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:00.426190 31446 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:34751
I20260810 22:40:00.426534 31519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:34751 every 8 connection(s)
I20260810 22:40:00.427191 31446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:40:00.428076 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 31446
I20260810 22:40:00.428205 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:40:00.428519 20265 external_mini_cluster.cc:1428] Setting key 85bd996a985cd5313f24724e9a99b4f1
I20260810 22:40:00.430500 31520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:00.437245 31520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8: Bootstrap starting.
Aug 10 22:40:00 dist-test-slave-1x4g krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:40:00.440120 31520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:00.441129 31520 log.cc:824] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8: Log is configured to *not* fsync() on all Append() calls
I20260810 22:40:00.443306 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:00.429889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.19.202.126:34751)
0810 22:40:00.430285 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:00.430301 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:00.430336 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:40:00.431050 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:00.431067 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:00.431394 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:00.431658 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:00.432500 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.433558 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.434652 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.435149 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.435276 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:00.438172 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:00.438178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:00.438193 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:00.438241 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:00.440195 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:00.440717 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:00.440722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:00.440727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:00.440909 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:00.441197 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:00.441199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:00.441200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:00.441631 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:00.441773 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:00.442144 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 22:40:00.442444 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":133,"threads_started":1}
I20260810 22:40:00.443370 31520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8: No bootstrap required, opened a new log
I20260810 22:40:00.446717 31520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a5bc8819cab446fa6e874d626f083b8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34751 } }
I20260810 22:40:00.446935 31520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:00.446979 31520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5bc8819cab446fa6e874d626f083b8, State: Initialized, Role: FOLLOWER
I20260810 22:40:00.447410 31520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a5bc8819cab446fa6e874d626f083b8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34751 } }
I20260810 22:40:00.447553 31520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:40:00.447598 31520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:40:00.447667 31520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:00.448591 31520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a5bc8819cab446fa6e874d626f083b8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34751 } }
I20260810 22:40:00.449000 31520 leader_election.cc:302] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a5bc8819cab446fa6e874d626f083b8; no voters:
I20260810 22:40:00.449337 31520 leader_election.cc:288] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:40:00.449471 31525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:40:00.449716 31525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5bc8819cab446fa6e874d626f083b8, State: Running, Role: LEADER
I20260810 22:40:00.450031 31525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a5bc8819cab446fa6e874d626f083b8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34751 } }
I20260810 22:40:00.450795 31520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:40:00.451647 31526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a5bc8819cab446fa6e874d626f083b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a5bc8819cab446fa6e874d626f083b8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34751 } } }
I20260810 22:40:00.451798 31526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [sys.catalog]: This master's current role is: LEADER
I20260810 22:40:00.452111 31527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a5bc8819cab446fa6e874d626f083b8. Latest consensus state: current_term: 1 leader_uuid: "6a5bc8819cab446fa6e874d626f083b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a5bc8819cab446fa6e874d626f083b8" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34751 } } }
I20260810 22:40:00.452207 31527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8 [sys.catalog]: This master's current role is: LEADER
I20260810 22:40:00.452256 31533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:40:00.455336 31533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:40:00.461308 31533 catalog_manager.cc:1347] Generated new cluster ID: 0ebbc68460f445b09b7bdb2ca44a1a19
I20260810 22:40:00.461385 31533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:40:00.468236 31533 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:40:00.469188 31533 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:40:00.480033 31533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8: Generated new TSK 0
I20260810 22:40:00.480875 31533 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:40:00.514022 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34751
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:40:00.625663 31548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:00.625914 31548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:00.625955 31548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:00.629876 31548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:00.629925 31548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:00.630017 31548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:40:00.635056 31548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:00.636102 31548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:00.637204 31548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:00.647050 31554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:00.647063 31553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:00.647154 31548 server_base.cc:1034] running on GCE node
W20260810 22:40:00.647043 31556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:00.647749 31548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:00.648384 31548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:00.649572 31548 hybrid_clock.cc:648] HybridClock initialized: now 1786401600649560 us; error 24 us; skew 500 ppm
Aug 10 22:40:00 dist-test-slave-1x4g krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401600, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:00.652509 31548 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:40:00.653429 31548 webserver.cc:460] Webserver started at http://127.19.202.65:44561/ using document root <none> and password file <none>
I20260810 22:40:00.653987 31548 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:00.654073 31548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:00.654302 31548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:40:00.656366 31548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "5ae975bc54654db481702a42e3affad9"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "546cbe2961df48f7c19f76fb79efa289"
server_key_iv: "bfb55f744ff8fdb1275592a1a1159f44"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.657032 31548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "5ae975bc54654db481702a42e3affad9"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "546cbe2961df48f7c19f76fb79efa289"
server_key_iv: "bfb55f744ff8fdb1275592a1a1159f44"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.661010 31548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 22:40:00.663534 31563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:00.664691 31548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:40:00.664858 31548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "5ae975bc54654db481702a42e3affad9"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "546cbe2961df48f7c19f76fb79efa289"
server_key_iv: "bfb55f744ff8fdb1275592a1a1159f44"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.664978 31548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:40:00.679133 31548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:00.682430 31548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:00.682636 31548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:00.683223 31548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:40:00.684087 31548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:40:00.684157 31548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:00.684227 31548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:40:00.684268 31548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:00.693259 31548 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:44529
I20260810 22:40:00.693279 31686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:44529 every 8 connection(s)
I20260810 22:40:00.694211 31548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:40:00.701493 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 31548
I20260810 22:40:00.701639 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:40:00.701960 20265 external_mini_cluster.cc:1428] Setting key 7e4694034bf562ddebb55cd153c588a3
Aug 10 22:40:00 dist-test-slave-1x4g krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401600, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:40:00.708907 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:00.696054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:49421 (local address 127.19.202.126:34751)
0810 22:40:00.696219 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:00.696223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:00.697064 (+ 841us) server_negotiation.cc:407] Connection header received
0810 22:40:00.698519 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:00.698522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:00.698581 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:00.698681 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:00.699871 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.700422 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.701608 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.701882 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.701987 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:00.704864 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:00.704867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:00.704869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:00.704893 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:00.706652 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:00.707164 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:00.707170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:00.707175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:00.707278 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:00.707604 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:00.707610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:00.707614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:00.707867 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:00.707972 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:00.708462 (+ 490us) server_negotiation.cc:299] Negotiation successful
0810 22:40:00.708624 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 22:40:00.709652 31689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:00.696405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34751 (local address 127.19.202.65:49421)
0810 22:40:00.696920 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:00.696964 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:00.698284 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:00.698854 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:00.698861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:00.699217 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:40:00.699726 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:00.699739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.700554 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.700558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:00.701479 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:00.701489 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.702043 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.702047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:00.702220 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:00.702817 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:40:00.702833 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:40:00.704676 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:40:00.706776 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:00.706781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:00.706793 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:00.707047 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:00.707370 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:00.707372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:00.707374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:00.707503 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:00.707968 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:40:00.707973 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:40:00.708199 (+ 226us) client_negotiation.cc:769] Sending connection context
0810 22:40:00.708343 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 22:40:00.708602 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":126,"threads_started":1}
I20260810 22:40:00.710958 31687 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34751
I20260810 22:40:00.711226 31687 heartbeater.cc:464] Registering TS with master...
I20260810 22:40:00.711838 31687 heartbeater.cc:511] Master 127.19.202.126:34751 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:40:00.713660 31481 ts_manager.cc:194] Registered new tserver with Master: 5ae975bc54654db481702a42e3affad9 (127.19.202.65:44529)
I20260810 22:40:00.714854 31481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:49421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:40:00.730962 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34751
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:40:00.843372 31693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:00.843633 31693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:00.843711 31693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:00.847764 31693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:00.847844 31693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:00.847931 31693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:40:00.853051 31693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:00.854056 31693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:00.855043 31693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:00.864446 31698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:00.864446 31699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:00.864487 31702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:00.864532 31693 server_base.cc:1034] running on GCE node
I20260810 22:40:00.865085 31693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:00.865680 31693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:00.866865 31693 hybrid_clock.cc:648] HybridClock initialized: now 1786401600866856 us; error 32 us; skew 500 ppm
Aug 10 22:40:00 dist-test-slave-1x4g krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401600, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:00.869627 31693 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:40:00.870404 31693 webserver.cc:460] Webserver started at http://127.19.202.66:44391/ using document root <none> and password file <none>
I20260810 22:40:00.870961 31693 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:00.871049 31693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:00.871279 31693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:40:00.873262 31693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "d38c8b796cd2413c94e21a1297d38712"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "0fb01289805a2085959a4cfe93f76ab2"
server_key_iv: "6550ec25972c9d2d6e6413054078563c"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.873831 31693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "d38c8b796cd2413c94e21a1297d38712"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "0fb01289805a2085959a4cfe93f76ab2"
server_key_iv: "6550ec25972c9d2d6e6413054078563c"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.877408 31693 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:40:00.879640 31711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:00.880873 31693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 22:40:00.880996 31693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "d38c8b796cd2413c94e21a1297d38712"
format_stamp: "Formatted at 2026-08-10 22:40:00 on dist-test-slave-1x4g"
server_key: "0fb01289805a2085959a4cfe93f76ab2"
server_key_iv: "6550ec25972c9d2d6e6413054078563c"
server_key_version: "encryptionkey@0"
I20260810 22:40:00.881102 31693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:40:00.911608 31693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:00.915759 31693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:00.915983 31693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:00.916826 31693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:40:00.918035 31693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:40:00.918093 31693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:00.918157 31693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:40:00.918190 31693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:00.928795 31693 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:42437
I20260810 22:40:00.928869 31837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:42437 every 8 connection(s)
I20260810 22:40:00.929929 31693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:40:00.937668 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 31693
I20260810 22:40:00.937794 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:40:00.938078 20265 external_mini_cluster.cc:1428] Setting key 259a38a3aa700aafbfb066d4b9dd4098
Aug 10 22:40:00 dist-test-slave-1x4g krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401600, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:40:00.947672 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:00.932078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:39105 (local address 127.19.202.126:34751)
0810 22:40:00.932231 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:00.932238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:00.933440 (+ 1202us) server_negotiation.cc:407] Connection header received
0810 22:40:00.935497 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:00.935500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:00.935555 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:00.935685 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:00.937479 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.938247 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.939853 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.940237 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.940352 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:00.943377 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:00.943380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:00.943384 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:00.943411 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:00.945215 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:00.945878 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:00.945881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:00.945883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:00.945985 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:00.946431 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:00.946434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:00.946436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:00.946625 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:00.946736 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:00.947365 (+ 629us) server_negotiation.cc:299] Negotiation successful
0810 22:40:00.947488 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:40:00.949126 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:00.932622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34751 (local address 127.19.202.66:39105)
0810 22:40:00.933238 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:00.933299 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:00.935254 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:00.935868 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:00.935880 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:00.936484 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:40:00.937289 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:00.937324 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.938403 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.938406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:00.939735 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:00.939746 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:00.940401 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:00.940407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:00.940621 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:00.941304 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:40:00.941329 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:40:00.943225 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:40:00.945347 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:00.945353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:00.945369 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:00.945761 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:00.946112 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:00.946118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:00.946122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:00.946307 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:00.946734 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:40:00.946741 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:40:00.947059 (+ 318us) client_negotiation.cc:769] Sending connection context
0810 22:40:00.947824 (+ 765us) client_negotiation.cc:240] Negotiation successful
0810 22:40:00.948177 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":189,"threads_started":1}
I20260810 22:40:00.950335 31838 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34751
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:40:00.950614 31838 heartbeater.cc:464] Registering TS with master...
I20260810 22:40:00.951112 31838 heartbeater.cc:511] Master 127.19.202.126:34751 requested a full tablet report, sending...
I20260810 22:40:00.952217 31481 ts_manager.cc:194] Registered new tserver with Master: d38c8b796cd2413c94e21a1297d38712 (127.19.202.66:42437)
I20260810 22:40:00.952713 31481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:39105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:40:00.967521 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34751
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:40:01.075479 31843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:01.075704 31843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:01.075744 31843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:01.079592 31843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:01.079658 31843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:01.079707 31843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:40:01.084460 31843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:01.085507 31843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:01.086448 31843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:01.095777 31849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:01.095789 31850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:01.095849 31843 server_base.cc:1034] running on GCE node
W20260810 22:40:01.095794 31854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:01.096498 31843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:01.097342 31843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:01.098531 31843 hybrid_clock.cc:648] HybridClock initialized: now 1786401601098452 us; error 97 us; skew 500 ppm
Aug 10 22:40:01 dist-test-slave-1x4g krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401601, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:01.101351 31843 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:40:01.102173 31843 webserver.cc:460] Webserver started at http://127.19.202.67:38145/ using document root <none> and password file <none>
I20260810 22:40:01.102766 31843 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:01.102828 31843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:01.103049 31843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:40:01.105150 31843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "d61670ea0e04465a96b8b75a9564d3b2"
format_stamp: "Formatted at 2026-08-10 22:40:01 on dist-test-slave-1x4g"
server_key: "217288d899e8a0cec71efb2b974ad1b8"
server_key_iv: "43d31b01f06990751c75d6fe6f6602ca"
server_key_version: "encryptionkey@0"
I20260810 22:40:01.105684 31843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "d61670ea0e04465a96b8b75a9564d3b2"
format_stamp: "Formatted at 2026-08-10 22:40:01 on dist-test-slave-1x4g"
server_key: "217288d899e8a0cec71efb2b974ad1b8"
server_key_iv: "43d31b01f06990751c75d6fe6f6602ca"
server_key_version: "encryptionkey@0"
I20260810 22:40:01.109256 31843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:40:01.111636 31862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:01.113022 31843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:40:01.113130 31843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "d61670ea0e04465a96b8b75a9564d3b2"
format_stamp: "Formatted at 2026-08-10 22:40:01 on dist-test-slave-1x4g"
server_key: "217288d899e8a0cec71efb2b974ad1b8"
server_key_iv: "43d31b01f06990751c75d6fe6f6602ca"
server_key_version: "encryptionkey@0"
I20260810 22:40:01.113229 31843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:40:01.122992 31843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:01.125808 31843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:01.126020 31843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:01.126598 31843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:40:01.127399 31843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:40:01.127456 31843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:01.127524 31843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:40:01.127566 31843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:01.136633 31843 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:42577
I20260810 22:40:01.136664 31988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:42577 every 8 connection(s)
I20260810 22:40:01.137594 31843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:40:01.143541 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 31843
I20260810 22:40:01.143671 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:40:01.143988 20265 external_mini_cluster.cc:1428] Setting key 0b58a2f2b3c28ae4ed34d101bd60fb92
Aug 10 22:40:01 dist-test-slave-1x4g krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401601, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:40:01.151755 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.139342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:33331 (local address 127.19.202.126:34751)
0810 22:40:01.139490 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.139494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.140312 (+ 818us) server_negotiation.cc:407] Connection header received
0810 22:40:01.141739 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.141742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.141788 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.141864 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:01.143016 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.143761 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.145244 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.145544 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.145628 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.148303 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:01.148307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:01.148310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:01.148344 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:01.149761 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:01.150268 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:01.150272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:01.150274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:01.150364 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:01.150735 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:01.150737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:01.150738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:01.150928 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:01.151000 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.151532 (+ 532us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.151641 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 22:40:01.152695 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.139705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34751 (local address 127.19.202.67:33331)
0810 22:40:01.140154 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:01.140196 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:01.141536 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:01.142034 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:01.142041 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:01.142403 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:40:01.142876 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.142889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.143892 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.143895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.145077 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.145088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.145708 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.145712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.145936 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.146630 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:40:01.146647 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:40:01.148121 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:40:01.149879 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:01.149885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:01.149899 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:01.150166 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:01.150505 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:01.150507 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:01.150508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:01.150649 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:01.151015 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:40:01.151019 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:40:01.151233 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 22:40:01.151437 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 22:40:01.151775 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":133,"threads_started":1}
I20260810 22:40:01.153878 31989 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34751
I20260810 22:40:01.154093 31989 heartbeater.cc:464] Registering TS with master...
I20260810 22:40:01.154546 31989 heartbeater.cc:511] Master 127.19.202.126:34751 requested a full tablet report, sending...
I20260810 22:40:01.155510 31481 ts_manager.cc:194] Registered new tserver with Master: d61670ea0e04465a96b8b75a9564d3b2 (127.19.202.67:42577)
I20260810 22:40:01.156023 31481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:33331
I20260810 22:40:01.157997 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:40:01.166731 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.159865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.19.202.126:34751)
0810 22:40:01.160259 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.160264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.160276 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:40:01.160308 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.160310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.160347 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.160421 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:01.161180 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.161657 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.162427 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.162627 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.162701 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.163963 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:01.163967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:01.163969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:01.163989 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:01.165411 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:01.165758 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:01.165760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:01.165761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:01.165833 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:01.166047 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:01.166049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:01.166050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:01.166266 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:01.166378 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.166535 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.166622 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:40:01.169507 31481 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:40:01.172571 31481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:40:01.187285 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.181294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:42437 (local address 127.0.0.1:59498)
0810 22:40:01.182161 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:01.182174 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:01.182255 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:01.183133 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:01.183136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:01.183155 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:01.183240 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.183250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.184486 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.184489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.185916 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.185926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.186715 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.186719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.186826 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.186864 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:40:01.187005 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 22:40:01.187128 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":71,"thread_start_us":101,"threads_started":1}
I20260810 22:40:01.187606 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.180916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44529 (local address 127.0.0.1:41716)
0810 22:40:01.181276 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:01.181328 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:01.181459 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:01.182503 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:01.182509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:01.182532 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:01.182693 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.182700 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.184148 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.184151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.185842 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.185853 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.187079 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.187082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.187250 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.187308 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 22:40:01.187393 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 22:40:01.187475 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":246,"thread_start_us":91,"threads_started":1}
I20260810 22:40:01.187290 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.181703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59498 (local address 127.19.202.66:42437)
0810 22:40:01.182648 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.182665 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.182689 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:40:01.182769 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.182775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.182972 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.183360 (+ 388us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:01.183495 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.184336 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.186013 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.186583 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.186712 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.186865 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.187039 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.187141 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":841,"thread_start_us":143,"threads_started":1}
I20260810 22:40:01.187983 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.181632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:42577 (local address 127.0.0.1:37512)
0810 22:40:01.182297 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:01.182309 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:01.182379 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:01.182972 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:01.182975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:01.182993 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:01.183117 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.183123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.184499 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.184502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.185773 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.185783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.187077 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.187081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.187268 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.187320 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 22:40:01.187384 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 22:40:01.187861 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":632,"thread_start_us":89,"threads_started":1}
I20260810 22:40:01.188514 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.181052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.19.202.65:44529)
0810 22:40:01.181985 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.182001 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.182026 (+ 25us) server_negotiation.cc:407] Connection header received
0810 22:40:01.182109 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.182114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.182331 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.182493 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:01.182818 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.183996 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.185955 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.186756 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.186867 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.187006 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.188245 (+ 1239us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.188340 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":132,"threads_started":1}
I20260810 22:40:01.187726 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.181933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.19.202.67:42577)
0810 22:40:01.182514 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.182525 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.182549 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:40:01.182620 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.182626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.182829 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.182987 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:01.183235 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.184298 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.185907 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.186756 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.186867 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.187006 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.187468 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.187564 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":123,"threads_started":1}
I20260810 22:40:01.190843 31914 tablet_service.cc:1467] Processing CreateTablet for tablet fa9e07a14b5b47b8931d1e68a0d5fa80 (DEFAULT_TABLE table=test-table [id=4640a2528320491e9950d50add48333b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:40:01.190879 31764 tablet_service.cc:1467] Processing CreateTablet for tablet fa9e07a14b5b47b8931d1e68a0d5fa80 (DEFAULT_TABLE table=test-table [id=4640a2528320491e9950d50add48333b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:40:01.191182 31617 tablet_service.cc:1467] Processing CreateTablet for tablet fa9e07a14b5b47b8931d1e68a0d5fa80 (DEFAULT_TABLE table=test-table [id=4640a2528320491e9950d50add48333b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:40:01.191880 31914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9e07a14b5b47b8931d1e68a0d5fa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:01.191944 31764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9e07a14b5b47b8931d1e68a0d5fa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:01.192077 31617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9e07a14b5b47b8931d1e68a0d5fa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:01.198514 32010 tablet_bootstrap.cc:492] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Bootstrap starting.
I20260810 22:40:01.200021 32011 tablet_bootstrap.cc:492] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9: Bootstrap starting.
I20260810 22:40:01.200093 32012 tablet_bootstrap.cc:492] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Bootstrap starting.
I20260810 22:40:01.205433 32012 tablet_bootstrap.cc:654] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:01.206600 32012 log.cc:824] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Log is configured to *not* fsync() on all Append() calls
I20260810 22:40:01.210227 32010 tablet_bootstrap.cc:654] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:01.210580 32011 tablet_bootstrap.cc:654] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:01.211503 32011 log.cc:824] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9: Log is configured to *not* fsync() on all Append() calls
I20260810 22:40:01.211639 32010 log.cc:824] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Log is configured to *not* fsync() on all Append() calls
I20260810 22:40:01.213783 32012 tablet_bootstrap.cc:492] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: No bootstrap required, opened a new log
I20260810 22:40:01.213783 32011 tablet_bootstrap.cc:492] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9: No bootstrap required, opened a new log
I20260810 22:40:01.213937 32010 tablet_bootstrap.cc:492] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: No bootstrap required, opened a new log
I20260810 22:40:01.214020 32012 ts_tablet_manager.cc:1397] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.003s
I20260810 22:40:01.214125 32011 ts_tablet_manager.cc:1397] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260810 22:40:01.214155 32010 ts_tablet_manager.cc:1397] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.005s
I20260810 22:40:01.218245 32012 raft_consensus.cc:348] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.218245 32011 raft_consensus.cc:348] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.218537 32012 raft_consensus.cc:374] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:01.218606 32012 raft_consensus.cc:729] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d38c8b796cd2413c94e21a1297d38712, State: Initialized, Role: FOLLOWER
I20260810 22:40:01.219128 32012 consensus_queue.cc:260] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.219441 32010 raft_consensus.cc:348] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.219725 32010 raft_consensus.cc:374] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:01.219679 32011 raft_consensus.cc:374] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:01.219792 32010 raft_consensus.cc:729] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61670ea0e04465a96b8b75a9564d3b2, State: Initialized, Role: FOLLOWER
I20260810 22:40:01.219841 32011 raft_consensus.cc:729] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae975bc54654db481702a42e3affad9, State: Initialized, Role: FOLLOWER
I20260810 22:40:01.220232 32012 ts_tablet_manager.cc:1428] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260810 22:40:01.220307 32010 consensus_queue.cc:260] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.220418 32011 consensus_queue.cc:260] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.220638 31838 heartbeater.cc:503] Master 127.19.202.126:34751 was elected leader, sending a full tablet report...
I20260810 22:40:01.221429 32010 ts_tablet_manager.cc:1428] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260810 22:40:01.221611 32011 ts_tablet_manager.cc:1428] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260810 22:40:01.221817 31989 heartbeater.cc:503] Master 127.19.202.126:34751 was elected leader, sending a full tablet report...
I20260810 22:40:01.222762 31687 heartbeater.cc:503] Master 127.19.202.126:34751 was elected leader, sending a full tablet report...
I20260810 22:40:01.338580 32017 raft_consensus.cc:482] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:40:01.338856 32017 raft_consensus.cc:504] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.340631 32017 leader_election.cc:288] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ae975bc54654db481702a42e3affad9 (127.19.202.65:44529), d61670ea0e04465a96b8b75a9564d3b2 (127.19.202.67:42577)
I20260810 22:40:01.345094 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.341169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:42577 (local address 127.19.202.66:49117)
0810 22:40:01.341632 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:01.341643 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:01.341763 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:01.342071 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:01.342074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:01.342089 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:01.342217 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.342223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.342944 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.342950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.343785 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.343794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.344488 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.344492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.344685 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.344710 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:40:01.344848 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 22:40:01.344920 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":146,"threads_started":1}
I20260810 22:40:01.345322 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.341434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:49117 (local address 127.19.202.67:42577)
0810 22:40:01.341567 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.341572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.341731 (+ 159us) server_negotiation.cc:407] Connection header received
0810 22:40:01.341919 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.341921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.341965 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.342049 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:01.342335 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.342829 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.343926 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.344327 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.344472 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.344529 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.344915 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.345144 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 22:40:01.346069 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.340886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44529 (local address 127.19.202.66:59405)
0810 22:40:01.341005 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:01.341023 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:01.341130 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:01.341605 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:01.341609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:01.341633 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:01.341793 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.341803 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.343070 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.343077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.344536 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:01.344552 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.345619 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.345624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:01.345770 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.345796 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 22:40:01.345889 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 22:40:01.345959 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 22:40:01.346040 31937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9e07a14b5b47b8931d1e68a0d5fa80" candidate_uuid: "d38c8b796cd2413c94e21a1297d38712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61670ea0e04465a96b8b75a9564d3b2" is_pre_election: true
I20260810 22:40:01.346243 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.341026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:59405 (local address 127.19.202.65:44529)
0810 22:40:01.341198 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.341204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.341225 (+ 21us) server_negotiation.cc:407] Connection header received
0810 22:40:01.341290 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.341295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.341355 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.341452 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:01.341942 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.342884 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.344704 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.345400 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.345520 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.345564 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.345928 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.346034 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 22:40:01.346452 31937 raft_consensus.cc:2454] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d38c8b796cd2413c94e21a1297d38712 in term 0.
I20260810 22:40:01.346830 31638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9e07a14b5b47b8931d1e68a0d5fa80" candidate_uuid: "d38c8b796cd2413c94e21a1297d38712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae975bc54654db481702a42e3affad9" is_pre_election: true
I20260810 22:40:01.346984 31712 leader_election.cc:302] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d38c8b796cd2413c94e21a1297d38712, d61670ea0e04465a96b8b75a9564d3b2; no voters:
I20260810 22:40:01.347155 31638 raft_consensus.cc:2454] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d38c8b796cd2413c94e21a1297d38712 in term 0.
I20260810 22:40:01.347352 32017 raft_consensus.cc:2781] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:40:01.347477 32017 raft_consensus.cc:482] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:40:01.347536 32017 raft_consensus.cc:3037] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:01.348677 32017 raft_consensus.cc:504] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.349082 32017 leader_election.cc:288] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [CANDIDATE]: Term 1 election: Requested vote from peers 5ae975bc54654db481702a42e3affad9 (127.19.202.65:44529), d61670ea0e04465a96b8b75a9564d3b2 (127.19.202.67:42577)
I20260810 22:40:01.349455 31638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9e07a14b5b47b8931d1e68a0d5fa80" candidate_uuid: "d38c8b796cd2413c94e21a1297d38712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae975bc54654db481702a42e3affad9"
I20260810 22:40:01.349498 31937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9e07a14b5b47b8931d1e68a0d5fa80" candidate_uuid: "d38c8b796cd2413c94e21a1297d38712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61670ea0e04465a96b8b75a9564d3b2"
I20260810 22:40:01.349606 31638 raft_consensus.cc:3037] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:01.349643 31937 raft_consensus.cc:3037] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:01.350673 31638 raft_consensus.cc:2454] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d38c8b796cd2413c94e21a1297d38712 in term 1.
I20260810 22:40:01.351011 31712 leader_election.cc:302] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae975bc54654db481702a42e3affad9, d38c8b796cd2413c94e21a1297d38712; no voters:
I20260810 22:40:01.351191 32017 raft_consensus.cc:2781] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:40:01.351418 31937 raft_consensus.cc:2454] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d38c8b796cd2413c94e21a1297d38712 in term 1.
I20260810 22:40:01.351472 32017 raft_consensus.cc:686] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 1 LEADER]: Becoming Leader. State: Replica: d38c8b796cd2413c94e21a1297d38712, State: Running, Role: LEADER
I20260810 22:40:01.352164 32017 consensus_queue.cc:237] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } }
I20260810 22:40:01.356074 31480 catalog_manager.cc:5495] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 reported cstate change: term changed from 0 to 1, leader changed from <none> to d38c8b796cd2413c94e21a1297d38712 (127.19.202.66). New cstate: current_term: 1 leader_uuid: "d38c8b796cd2413c94e21a1297d38712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d38c8b796cd2413c94e21a1297d38712" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 42437 } health_report { overall_health: HEALTHY } } }
I20260810 22:40:01.378381 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.373835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.19.202.66:42437)
0810 22:40:01.373986 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:01.373992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:01.374012 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:40:01.374147 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:01.374152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:01.374247 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:01.374375 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:40:01.374606 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.375513 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.376360 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:01.376834 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:01.376965 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:01.377239 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:40:01.377861 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:40:01.378012 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:01.378067 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 22:40:01.378149 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:40:01.384693 31937 raft_consensus.cc:1261] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 1 FOLLOWER]: Refusing update from remote peer d38c8b796cd2413c94e21a1297d38712: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:40:01.384693 31638 raft_consensus.cc:1261] T fa9e07a14b5b47b8931d1e68a0d5fa80 P 5ae975bc54654db481702a42e3affad9 [term 1 FOLLOWER]: Refusing update from remote peer d38c8b796cd2413c94e21a1297d38712: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:40:01.385550 32017 consensus_queue.cc:1035] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d61670ea0e04465a96b8b75a9564d3b2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:40:01.385807 32022 consensus_queue.cc:1035] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae975bc54654db481702a42e3affad9" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 44529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 22:40:01.388435 31990 tablet.cc:2369] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:40:01.394487 32026 mvcc.cc:205] Tried to move back new op lower bound from 7317100959263559680 to 7317100959141904384. Current Snapshot: MvccSnapshot[applied={T|T < 7317100959263559680}]
I20260810 22:40:01.395714 32027 mvcc.cc:205] Tried to move back new op lower bound from 7317100959263559680 to 7317100959141904384. Current Snapshot: MvccSnapshot[applied={T|T < 7317100959263559680}]
I20260810 22:40:01.399650 32028 mvcc.cc:205] Tried to move back new op lower bound from 7317100959263559680 to 7317100959141904384. Current Snapshot: MvccSnapshot[applied={T|T < 7317100959263559680}]
I20260810 22:40:01.405397 31481 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:40:01.405614 31481 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:40:01.408048 31481 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a5bc8819cab446fa6e874d626f083b8: Sending DeleteTablet for 3 replicas of tablet fa9e07a14b5b47b8931d1e68a0d5fa80
I20260810 22:40:01.409022 31764 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9e07a14b5b47b8931d1e68a0d5fa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:40:01 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:59498
I20260810 22:40:01.409104 31617 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9e07a14b5b47b8931d1e68a0d5fa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:40:01 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:41716
I20260810 22:40:01.409587 31914 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9e07a14b5b47b8931d1e68a0d5fa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:40:01 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:37512
I20260810 22:40:01.409792 32037 tablet_replica.cc:330] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: stopping tablet replica
I20260810 22:40:01.410188 32037 raft_consensus.cc:2229] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:40:01.410692 32037 raft_consensus.cc:2258] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:40:01.410983 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 31548
I20260810 22:40:01.412544 32037 ts_tablet_manager.cc:1905] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:40:01.415530 32037 ts_tablet_manager.cc:1918] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:40:01.415644 32037 log.cc:1196] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/fa9e07a14b5b47b8931d1e68a0d5fa80
I20260810 22:40:01.415978 32037 ts_tablet_manager.cc:1939] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d38c8b796cd2413c94e21a1297d38712: Deleting consensus metadata
I20260810 22:40:01.416455 32038 tablet_replica.cc:330] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: stopping tablet replica
I20260810 22:40:01.416994 32038 raft_consensus.cc:2229] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:40:01.417481 32038 raft_consensus.cc:2258] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:40:01.419298 32038 ts_tablet_manager.cc:1905] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:40:01.421080 31466 catalog_manager.cc:4841] TS d38c8b796cd2413c94e21a1297d38712 (127.19.202.66:42437): tablet fa9e07a14b5b47b8931d1e68a0d5fa80 (table test-table [id=4640a2528320491e9950d50add48333b]) successfully deleted
I20260810 22:40:01.422955 32038 ts_tablet_manager.cc:1918] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:40:01.423074 32038 log.cc:1196] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/fa9e07a14b5b47b8931d1e68a0d5fa80
I20260810 22:40:01.423460 32038 ts_tablet_manager.cc:1939] T fa9e07a14b5b47b8931d1e68a0d5fa80 P d61670ea0e04465a96b8b75a9564d3b2: Deleting consensus metadata
W20260810 22:40:01.423702 31463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:44529 (error 108)
I20260810 22:40:01.424877 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:01.424550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:44529 (local address 127.0.0.1:41722)
0810 22:40:01.424693 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:01.424766 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 22:40:01.424842 31463 catalog_manager.cc:4841] TS d61670ea0e04465a96b8b75a9564d3b2 (127.19.202.67:42577): tablet fa9e07a14b5b47b8931d1e68a0d5fa80 (table test-table [id=4640a2528320491e9950d50add48333b]) successfully deleted
I20260810 22:40:01.425004 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 31693
W20260810 22:40:01.425063 31463 catalog_manager.cc:4584] TS 5ae975bc54654db481702a42e3affad9 (127.19.202.65:44529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa9e07a14b5b47b8931d1e68a0d5fa80: Network error: Client connection negotiation failed: client connection to 127.19.202.65:44529: connect: Connection refused (error 111)
I20260810 22:40:01.434572 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 31843
I20260810 22:40:01.442693 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 31446
2026-08-10T22:40:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2366 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:40:01 dist-test-slave-1x4g krb5kdc[32043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:40:01 dist-test-slave-1x4g krb5kdc[32043](info): set up 2 sockets
Aug 10 22:40:01 dist-test-slave-1x4g krb5kdc[32043](info): commencing operation
krb5kdc: starting...
W20260810 22:40:02.497042 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:40:02 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:40:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/rangeradmin_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/rangeradmin_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/rangerlookup_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/rangerlookup_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
I20260810 22:40:02.655396 20265 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskx9HXXp/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/postgres -l logfile start
2026-08-10 22:40:05.015 UTC [32076] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 22:40:05.015 UTC [32076] LOG: listening on IPv4 address "127.19.202.84", port 55647
2026-08-10 22:40:05.018 UTC [32076] LOG: listening on Unix socket "/tmp/.s.PGSQL.55647"
2026-08-10 22:40:05.038 UTC [32079] LOG: database system was shut down at 2026-08-10 22:40:03 UTC
2026-08-10 22:40:05.041 UTC [32076] LOG: database system is ready to accept connections
I20260810 22:40:06.009320 20265 mini_postgres.cc:96] Postgres bound to 55647
2026-08-10 22:40:06.016 UTC [32088] FATAL: database "slave" does not exist
127.19.202.84:55647 - accepting connections
I20260810 22:40:06.017318 20265 mini_ranger.cc:161] Starting Ranger...
I20260810 22:40:06.032291 20265 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 22:40:06.480834 20265 mini_ranger.cc:87] Created ranger Postgres database
I20260810 22:40:06.480944 20265 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-admin
2026-08-10 22:40:06,760 [I] DB FLAVOR :POSTGRES
2026-08-10 22:40:06,761 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 22:40:06,761 [I] Checking connection..
2026-08-10 22:40:06,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 22:40:07,011 [I] Checking connection passed.
2026-08-10 22:40:07,011 [I] --------- Verifying version history table ---------
2026-08-10 22:40:07,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:40:07,265 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 22:40:07,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:40:07,493 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 22:40:07,494 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 22:40:07,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 22:40:07.726 UTC [32161] WARNING: there is no transaction in progress
2026-08-10 22:40:07,733 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 22:40:07,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:40:07,975 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 22:40:07,975 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 22:40:07,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 22:40:08,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 22:40:08,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-10 22:40:08,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 22:40:08,951 [I] Table x_portal_user does not exist in database ranger
2026-08-10 22:40:08,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 22:40:09,208 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 22:40:09,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 22:40:09,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 22:40:09,698 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 22:40:09,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 22:40:10.619 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.635 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.651 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.664 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.679 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.696 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.749 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.755 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.769 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.779 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.789 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.801 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.810 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.817 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.823 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:10.830 UTC [32278] WARNING: there is no transaction in progress
2026-08-10 22:40:11,448 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 22:40:11,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 22:40:11,660 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 22:40:11,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 22:40:11,896 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 22:40:11,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 22:40:12,148 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 22:40:12,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 22:40:12,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 22:40:12,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-10 22:40:12,868 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 22:40:12,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 22:40:13,085 [I] DB_PATCHES have already been applied
I20260810 22:40:13.091475 20265 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-admin:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskx9HXXp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 22:40:13.091531 20265 mini_ranger.cc:193] Using host: 127.19.202.84
I20260810 22:40:13.096021 20265 mini_ranger.cc:227] Ranger admin URL: http://127.19.202.84:57965
Aug 10, 2026 10:40:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 10:40:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 10:40:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 10:40:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 10:40:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 10:40:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 10:40:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 22:40:14.248606 20265 mini_ranger.cc:160] Time spent starting Ranger: real 8.231s user 0.000s sys 0.012s
Aug 10, 2026 10:40:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 10:40:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 10:40:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 10:40:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 10:40:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 22:40:35.671625 20265 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/rangerkms_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/rangerkms_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.202.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.202.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 22:40:35 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401635, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 22:40:35.766788 20265 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 22:40:35.779933 20265 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 22:40:36.607422 20265 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 22:40:36.607527 20265 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms
I20260810 22:40:36.607635 20265 mini_ranger_kms.cc:227] Using postgres at 127.19.202.84:55647
2026-08-10 22:40:37,177 [I] DB FLAVOR :POSTGRES
2026-08-10 22:40:37,177 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 22:40:37,177 [I] Checking connection
2026-08-10 22:40:37,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 22:40:37,494 [I] connection success
2026-08-10 22:40:37,494 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 22:40:37,494 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 22:40:37,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 22:40:37,739 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 22:40:37,740 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 22:40:37,740 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 22:40:37,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:55647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 22:40:38,009 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 22:40:38.470963 20265 mini_ranger_kms.cc:310] Created kms service
I20260810 22:40:38.667040 20265 mini_ranger_kms.cc:326] Created kudu user
I20260810 22:40:38.709090 20265 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 22:40:38 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401635, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:40:39.120235 20265 mini_ranger_kms.cc:384] Added ranger policy
I20260810 22:40:39.120436 20265 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms:/tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 22:40:39.120471 20265 mini_ranger_kms.cc:240] Using host: 127.19.202.84
I20260810 22:40:39.125900 20265 mini_ranger_kms.cc:276] Ranger KMS PID: 32478
I20260810 22:40:39.126060 20265 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.202.84:39587
Aug 10, 2026 10:40:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 10:40:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 10:40:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 10:40:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 10:40:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 10:40:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 22:40:41.516827 20265 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.750s user 0.002s sys 0.539s
I20260810 22:40:41.516995 20265 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 22:40:41.517127 20265 mini_ranger_kms.cc:401] 127.19.202.84:39587/kms/v1/keys
Aug 10 22:40:41 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786401641, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.202.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 22:40:42 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401641, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.202.84@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
Aug 10 22:40:42 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401641, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.202.84@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
Aug 10 22:40:45 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:40:45.287580 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38665
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:38665
--ranger_config_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:39587/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:40:45.472707 32535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:45.473024 32535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:45.473095 32535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:45.476863 32535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:40:45.476927 32535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:45.476948 32535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:45.476963 32535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:40:45.477011 32535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:40:45.482542 32535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.202.84:39587/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:38665
--ranger_config_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:45.483652 32535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:45.484678 32535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:45.494028 32541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:45.494087 32542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:45.494194 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:45.494159 32535 server_base.cc:1034] running on GCE node
I20260810 22:40:45.494875 32535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:45.495905 32535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:45.497093 32535 hybrid_clock.cc:648] HybridClock initialized: now 1786401645497079 us; error 31 us; skew 500 ppm
Aug 10 22:40:45 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401645, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:45.500175 32535 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:40:45.501116 32535 webserver.cc:460] Webserver started at http://127.19.202.126:39709/ using document root <none> and password file <none>
I20260810 22:40:45.501662 32535 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:45.501744 32535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:45.501976 32535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:40:45 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401645, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:40:45.564527 32535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "89dcadaa324e4e33be90afb3a94546b7"
format_stamp: "Formatted at 2026-08-10 22:40:45 on dist-test-slave-1x4g"
server_key: "41b3c90d91ab07b1184b381fc4b0a87f"
server_key_iv: "87dab9788e8c3158b23a94d0bdc60aed"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:45.565335 32535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "89dcadaa324e4e33be90afb3a94546b7"
format_stamp: "Formatted at 2026-08-10 22:40:45 on dist-test-slave-1x4g"
server_key: "41b3c90d91ab07b1184b381fc4b0a87f"
server_key_iv: "87dab9788e8c3158b23a94d0bdc60aed"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:45.569798 32535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 22:40:45.586997 32552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:45.588441 32535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 22:40:45.588598 32535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "89dcadaa324e4e33be90afb3a94546b7"
format_stamp: "Formatted at 2026-08-10 22:40:45 on dist-test-slave-1x4g"
server_key: "41b3c90d91ab07b1184b381fc4b0a87f"
server_key_iv: "87dab9788e8c3158b23a94d0bdc60aed"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:45.588747 32535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:40:45.616387 32535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:45.619637 32535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:45.619875 32535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:45.630728 32535 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:38665
I20260810 22:40:45.630822 32612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:38665 every 8 connection(s)
I20260810 22:40:45.631839 32535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:40:45.634883 32613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:45.636762 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 32535
I20260810 22:40:45.636909 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:40:45.637204 20265 external_mini_cluster.cc:1428] Setting key 6b99e327bb812d9b32611235ee9a8255
I20260810 22:40:45.642153 32613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7: Bootstrap starting.
I20260810 22:40:45.644922 32613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:45.646011 32613 log.cc:824] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7: Log is configured to *not* fsync() on all Append() calls
Aug 10 22:40:45 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:40:45.648198 32613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7: No bootstrap required, opened a new log
I20260810 22:40:45.652936 32613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dcadaa324e4e33be90afb3a94546b7" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38665 } }
I20260810 22:40:45.653201 32613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:45.653295 32613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89dcadaa324e4e33be90afb3a94546b7, State: Initialized, Role: FOLLOWER
I20260810 22:40:45.653864 32613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dcadaa324e4e33be90afb3a94546b7" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38665 } }
I20260810 22:40:45.654057 32613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:40:45.654166 32613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:40:45.654275 32613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:45.655323 32616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:45.638634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.19.202.126:38665)
0810 22:40:45.639100 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:45.639127 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:45.639170 (+ 43us) server_negotiation.cc:407] Connection header received
0810 22:40:45.640211 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:45.640235 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:45.640663 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:45.641058 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:45.641977 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:45.643451 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:45.644696 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:45.645478 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:45.645641 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:45.648741 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:45.648748 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:45.648764 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:45.648828 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:45.651774 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:45.652270 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:45.652276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:45.652282 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:45.652428 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:45.652815 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:45.652818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:45.652820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:45.653316 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:45.653548 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:45.653827 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 22:40:45.654267 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":139,"threads_started":1}
I20260810 22:40:45.655445 32613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dcadaa324e4e33be90afb3a94546b7" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38665 } }
I20260810 22:40:45.655838 32613 leader_election.cc:302] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89dcadaa324e4e33be90afb3a94546b7; no voters:
I20260810 22:40:45.656160 32613 leader_election.cc:288] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:40:45.656472 32621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:40:45.656903 32621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [term 1 LEADER]: Becoming Leader. State: Replica: 89dcadaa324e4e33be90afb3a94546b7, State: Running, Role: LEADER
I20260810 22:40:45.657266 32621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dcadaa324e4e33be90afb3a94546b7" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38665 } }
I20260810 22:40:45.658773 32613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:40:45.659053 32622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89dcadaa324e4e33be90afb3a94546b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dcadaa324e4e33be90afb3a94546b7" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38665 } } }
I20260810 22:40:45.659163 32622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [sys.catalog]: This master's current role is: LEADER
I20260810 22:40:45.659520 32623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89dcadaa324e4e33be90afb3a94546b7. Latest consensus state: current_term: 1 leader_uuid: "89dcadaa324e4e33be90afb3a94546b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dcadaa324e4e33be90afb3a94546b7" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38665 } } }
I20260810 22:40:45.659680 32623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7 [sys.catalog]: This master's current role is: LEADER
I20260810 22:40:45.660770 32625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:40:45.664059 32625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:40:45.664638 32613 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 22:40:45.674535 32625 catalog_manager.cc:1347] Generated new cluster ID: a341dbd314c047b2876f5da1a5e093dd
I20260810 22:40:45.674620 32625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:40:45.703930 32625 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:40:45.704977 32625 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:40:45.715677 32625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7: Generated new TSK 0
I20260810 22:40:45.716348 32625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:40:47.008869 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38665
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:40:47.347786 32663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:47.348205 32663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:47.348344 32663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:47.354367 32663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:47.354445 32663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:47.354605 32663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:40:47.363471 32663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.202.84:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:47.365592 32663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:47.367451 32663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:47.389099 32669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:47.389870 32672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:47.390035 32670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:47.390625 32663 server_base.cc:1034] running on GCE node
I20260810 22:40:47.391513 32663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:47.392917 32663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:47.394196 32663 hybrid_clock.cc:648] HybridClock initialized: now 1786401647394151 us; error 72 us; skew 500 ppm
Aug 10 22:40:47 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401647, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:47.401521 32663 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:40:47.403003 32663 webserver.cc:460] Webserver started at http://127.19.202.65:35979/ using document root <none> and password file <none>
I20260810 22:40:47.403951 32663 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:47.404065 32663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:47.404528 32663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:40:47 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401647, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:40:47.441385 32663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "8455c904fc3341c180c046ca8e9ab868"
format_stamp: "Formatted at 2026-08-10 22:40:47 on dist-test-slave-1x4g"
server_key: "e8bbd55496528138a10d5d6cb957f422"
server_key_iv: "9f460ee0443d3e87c64bea43f08982b5"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:47.443115 32663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "8455c904fc3341c180c046ca8e9ab868"
format_stamp: "Formatted at 2026-08-10 22:40:47 on dist-test-slave-1x4g"
server_key: "e8bbd55496528138a10d5d6cb957f422"
server_key_iv: "9f460ee0443d3e87c64bea43f08982b5"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:47.453549 32663 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.005s
I20260810 22:40:47.484061 32682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:47.486960 32663 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.000s
I20260810 22:40:47.487290 32663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "8455c904fc3341c180c046ca8e9ab868"
format_stamp: "Formatted at 2026-08-10 22:40:47 on dist-test-slave-1x4g"
server_key: "e8bbd55496528138a10d5d6cb957f422"
server_key_iv: "9f460ee0443d3e87c64bea43f08982b5"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:47.487632 32663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:40:47.539717 32663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:47.546690 32663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:47.547163 32663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:47.548602 32663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:40:47.550585 32663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:40:47.550776 32663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 22:40:47.550964 32663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:40:47.551103 32663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:47.602635 32663 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:42855
I20260810 22:40:47.604889 32663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:40:47.608608 336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:42855 every 8 connection(s)
I20260810 22:40:47.613341 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 32663
I20260810 22:40:47.613576 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:40:47.614262 20265 external_mini_cluster.cc:1428] Setting key c291ff7ebc78ab128b277746937dde08
Aug 10 22:40:47 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401647, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:40:47.641494 340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:47.613118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:44393 (local address 127.19.202.126:38665)
0810 22:40:47.620674 (+ 7556us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:47.620681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:47.620715 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:40:47.620819 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:47.620823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:47.620908 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:47.621068 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:47.624179 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:47.625467 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:47.628276 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:47.628824 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:47.628951 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:47.634556 (+ 5605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:47.634570 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:47.634574 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:47.634612 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:47.637846 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:47.638721 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:47.638735 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:47.638737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:47.638874 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:47.639446 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:47.639449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:47.639451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:47.639755 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:47.639842 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:47.641058 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0810 22:40:47.641260 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7431,"thread_start_us":211,"threads_started":1}
I20260810 22:40:47.644708 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:47.613883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38665 (local address 127.19.202.65:44393)
0810 22:40:47.617453 (+ 3570us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:47.617561 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:47.619978 (+ 2417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:47.621589 (+ 1611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:47.621607 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:47.622471 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:40:47.623836 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:47.623899 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:47.625708 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:47.625711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:47.628073 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:47.628085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:47.629272 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:47.629277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:47.629780 (+ 503us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:47.631127 (+ 1347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:40:47.631161 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:40:47.634278 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:40:47.638011 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:47.638032 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:47.638048 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:47.638540 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:47.639035 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:47.639038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:47.639040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:47.639301 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:47.639932 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:40:47.639952 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:40:47.640512 (+ 560us) client_negotiation.cc:769] Sending connection context
0810 22:40:47.642276 (+ 1764us) client_negotiation.cc:240] Negotiation successful
0810 22:40:47.642791 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3156,"thread_start_us":298,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:40:47.647501 337 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38665
I20260810 22:40:47.647974 337 heartbeater.cc:464] Registering TS with master...
I20260810 22:40:47.649010 337 heartbeater.cc:511] Master 127.19.202.126:38665 requested a full tablet report, sending...
I20260810 22:40:47.651764 32572 ts_manager.cc:194] Registered new tserver with Master: 8455c904fc3341c180c046ca8e9ab868 (127.19.202.65:42855)
I20260810 22:40:47.654031 32572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:44393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:40:47.701339 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38665
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:40:47 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786401647, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 22:40:48.087662 346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:48.088131 346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:48.088289 346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:48.096868 346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:48.096982 346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:48.097132 346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:40:48.110752 346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.202.84:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:48.113643 346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:48.116389 346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:48.141069 353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:48.141959 346 server_base.cc:1034] running on GCE node
W20260810 22:40:48.144827 356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:48.145175 352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:48.146138 346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:48.147151 346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:48.150476 346 hybrid_clock.cc:648] HybridClock initialized: now 1786401648150375 us; error 110 us; skew 500 ppm
Aug 10 22:40:48 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401648, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:48.156320 346 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:40:48.158100 346 webserver.cc:460] Webserver started at http://127.19.202.66:41061/ using document root <none> and password file <none>
I20260810 22:40:48.159466 346 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:48.159722 346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:48.160342 346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:40:48 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401648, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:40:48.276918 346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa"
format_stamp: "Formatted at 2026-08-10 22:40:48 on dist-test-slave-1x4g"
server_key: "c2f2f19eb0aa2375c657c5502ff3faff"
server_key_iv: "9c9889e84962d24c8b6dcc0a6402ace1"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:48.279434 346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa"
format_stamp: "Formatted at 2026-08-10 22:40:48 on dist-test-slave-1x4g"
server_key: "c2f2f19eb0aa2375c657c5502ff3faff"
server_key_iv: "9c9889e84962d24c8b6dcc0a6402ace1"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:48.291706 346 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.003s sys 0.003s
I20260810 22:40:48.333644 364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:48.336489 346 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.005s sys 0.000s
I20260810 22:40:48.336663 346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa"
format_stamp: "Formatted at 2026-08-10 22:40:48 on dist-test-slave-1x4g"
server_key: "c2f2f19eb0aa2375c657c5502ff3faff"
server_key_iv: "9c9889e84962d24c8b6dcc0a6402ace1"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:48.336994 346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:40:48.381366 346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:48.387945 346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:48.388316 346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:48.389628 346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:40:48.391396 346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:40:48.391501 346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:48.391628 346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:40:48.391685 346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:48.429247 346 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:32889
I20260810 22:40:48.431746 346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:40:48.433487 487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:32889 every 8 connection(s)
I20260810 22:40:48.442001 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 346
I20260810 22:40:48.442226 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:40:48.442812 20265 external_mini_cluster.cc:1428] Setting key e8d8dbb49a80095fec7def7a05d9d0d5
Aug 10 22:40:48 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401648, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:40:48.464592 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:48.435455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:59777 (local address 127.19.202.126:38665)
0810 22:40:48.435872 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:48.435887 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:48.438246 (+ 2359us) server_negotiation.cc:407] Connection header received
0810 22:40:48.440732 (+ 2486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:48.440737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:48.440865 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:48.441084 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:48.444430 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:48.446026 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:48.449202 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:48.449770 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:48.449925 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:48.457187 (+ 7262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:48.457201 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:48.457205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:48.457254 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:48.460585 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:48.461598 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:48.461601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:48.461612 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:48.461719 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:48.462386 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:48.462399 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:48.462401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:48.462798 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:48.462946 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:48.464080 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0810 22:40:48.464293 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":286,"thread_start_us":152,"threads_started":1}
I20260810 22:40:48.467839 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:48.436940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38665 (local address 127.19.202.66:59777)
0810 22:40:48.437956 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:48.438070 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:48.440292 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:48.441942 (+ 1650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:48.441951 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:48.442693 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:40:48.444098 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:48.444142 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:48.446311 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:48.446314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:48.448989 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:48.449001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:48.450233 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:48.450238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:48.450678 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:48.453655 (+ 2977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:40:48.453675 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:40:48.456854 (+ 3179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:40:48.460871 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:48.460884 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:48.460910 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:48.461421 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:48.461889 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:48.461892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:48.461903 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:48.462221 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:48.463072 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:40:48.463081 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:40:48.463630 (+ 549us) client_negotiation.cc:769] Sending connection context
0810 22:40:48.465405 (+ 1775us) client_negotiation.cc:240] Negotiation successful
0810 22:40:48.466038 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":306,"threads_started":1}
I20260810 22:40:48.470294 488 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38665
I20260810 22:40:48.470754 488 heartbeater.cc:464] Registering TS with master...
I20260810 22:40:48.471746 488 heartbeater.cc:511] Master 127.19.202.126:38665 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:40:48.473472 32572 ts_manager.cc:194] Registered new tserver with Master: 843d9b5c8bd9484aa24ae0cd7b7642aa (127.19.202.66:32889)
I20260810 22:40:48.474723 32572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:59777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:40:48.504045 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38665
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:40:48.658874 337 heartbeater.cc:503] Master 127.19.202.126:38665 was elected leader, sending a full tablet report...
W20260810 22:40:48.835793 495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:40:48.836457 495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:40:48.836675 495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:40:48.845536 495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:40:48.845672 495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:40:48.845875 495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:40:48.859236 495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.202.84:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:40:48.861702 495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:40:48.863824 495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:40:48.885643 502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:48.886374 495 server_base.cc:1034] running on GCE node
W20260810 22:40:48.886391 501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:40:48.886425 504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:40:48.887387 495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:40:48.888615 495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:40:48.890305 495 hybrid_clock.cc:648] HybridClock initialized: now 1786401648890206 us; error 133 us; skew 500 ppm
Aug 10 22:40:48 dist-test-slave-1x4g krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401648, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:40:48.899528 495 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:40:48.901482 495 webserver.cc:460] Webserver started at http://127.19.202.67:35617/ using document root <none> and password file <none>
I20260810 22:40:48.902585 495 fs_manager.cc:359] Metadata directory not provided
I20260810 22:40:48.902736 495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:40:48.903148 495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:40:48 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401648, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:40:48.931743 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "7d4afde085ce4274a75fecfe19e098b9"
format_stamp: "Formatted at 2026-08-10 22:40:48 on dist-test-slave-1x4g"
server_key: "8615f1859b5e76b5bf67ab99123b6407"
server_key_iv: "173c5511739cd52dc9da4c47741f943d"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:48.933652 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "7d4afde085ce4274a75fecfe19e098b9"
format_stamp: "Formatted at 2026-08-10 22:40:48 on dist-test-slave-1x4g"
server_key: "8615f1859b5e76b5bf67ab99123b6407"
server_key_iv: "173c5511739cd52dc9da4c47741f943d"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:48.943006 495 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260810 22:40:48.984637 511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:48.987483 495 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.003s
I20260810 22:40:48.987680 495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "7d4afde085ce4274a75fecfe19e098b9"
format_stamp: "Formatted at 2026-08-10 22:40:48 on dist-test-slave-1x4g"
server_key: "8615f1859b5e76b5bf67ab99123b6407"
server_key_iv: "173c5511739cd52dc9da4c47741f943d"
server_key_version: "kuduclusterkey@0"
I20260810 22:40:48.987882 495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 22:40:49 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401647, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:40:49.068573 495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:40:49.081310 495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:40:49.081698 495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:40:49.082903 495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:40:49.084543 495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:40:49.084637 495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:49.084715 495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:40:49.084769 495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:40:49.141845 495 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:45977
I20260810 22:40:49.141988 633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:45977 every 8 connection(s)
I20260810 22:40:49.144187 495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:40:49.152117 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 495
I20260810 22:40:49.152330 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:40:49.152910 20265 external_mini_cluster.cc:1428] Setting key ac3fdbafb1745c9f954d81b338114e2d
Aug 10 22:40:49 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401648, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:40:49.177124 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.148224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:48463 (local address 127.19.202.126:38665)
0810 22:40:49.148695 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.148700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.150745 (+ 2045us) server_negotiation.cc:407] Connection header received
0810 22:40:49.153185 (+ 2440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.153189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.153319 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.153415 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:49.157646 (+ 4231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.159052 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.161984 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.163041 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.163217 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.169005 (+ 5788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:49.169010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:49.169013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:49.169086 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:49.172624 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:49.173889 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:49.173894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:49.173896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:49.174080 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:49.174815 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:49.174818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:49.174820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:49.175182 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:49.175355 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.176586 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.176817 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":122,"threads_started":1}
I20260810 22:40:49.179728 637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.149273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38665 (local address 127.19.202.67:48463)
0810 22:40:49.150472 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:49.150580 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:49.152682 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:49.154705 (+ 2023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:49.154727 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:49.155903 (+ 1176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:40:49.157389 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.157433 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.159285 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.159288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.161732 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.161754 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.163292 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.163297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.163781 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.165140 (+ 1359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:40:49.165177 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:40:49.168646 (+ 3469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:40:49.172841 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:49.172848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:49.172883 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:49.173694 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:49.174246 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:40:49.174262 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:40:49.174264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:40:49.174652 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:40:49.175382 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:40:49.175390 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:40:49.176023 (+ 633us) client_negotiation.cc:769] Sending connection context
0810 22:40:49.176490 (+ 467us) client_negotiation.cc:240] Negotiation successful
0810 22:40:49.177589 (+ 1099us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":335,"threads_started":1}
I20260810 22:40:49.182231 634 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38665
I20260810 22:40:49.182639 634 heartbeater.cc:464] Registering TS with master...
I20260810 22:40:49.183688 634 heartbeater.cc:511] Master 127.19.202.126:38665 requested a full tablet report, sending...
I20260810 22:40:49.185504 32572 ts_manager.cc:194] Registered new tserver with Master: 7d4afde085ce4274a75fecfe19e098b9 (127.19.202.67:45977)
I20260810 22:40:49.186718 32572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:48463
I20260810 22:40:49.196445 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:40:49.216284 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.200539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.19.202.126:38665)
0810 22:40:49.200739 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.200864 (+ 125us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.204353 (+ 3489us) server_negotiation.cc:407] Connection header received
0810 22:40:49.204636 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.204640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.204744 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.204924 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:40:49.206238 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.207044 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.208302 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.208647 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.208855 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.211098 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:40:49.211101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:40:49.211104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:40:49.211139 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:40:49.213456 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:49.214118 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:49.214124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:49.214126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:49.214291 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:40:49.214742 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:40:49.214754 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:40:49.214755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:40:49.215068 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:40:49.215308 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.215673 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.215878 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 22:40:49.220701 32572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:40:49.225978 32572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:40:49.255741 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.241384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:42855 (local address 127.0.0.1:40992)
0810 22:40:49.243504 (+ 2120us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:49.243585 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:49.243788 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:49.245946 (+ 2158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:49.245951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:49.245984 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:49.246432 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.246439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.249866 (+ 3427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.249870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.251529 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.251540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.254756 (+ 3216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.254761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.254971 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.255062 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 22:40:49.255221 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 22:40:49.255448 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"mutex_wait_us":50,"spinlock_wait_cycles":324352,"thread_start_us":198,"threads_started":1}
I20260810 22:40:49.256018 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.241900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45977 (local address 127.0.0.1:46730)
0810 22:40:49.244279 (+ 2379us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:49.244322 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:49.244555 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:49.245954 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:49.245976 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:49.246151 (+ 175us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:49.246439 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.246447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.248592 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.248595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.250552 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.250563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.255305 (+ 4742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.255310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.255472 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.255505 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 22:40:49.255615 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 22:40:49.255705 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2276,"mutex_wait_us":30,"thread_start_us":227,"threads_started":1}
I20260810 22:40:49.257342 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.243079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46730 (local address 127.19.202.67:45977)
0810 22:40:49.245173 (+ 2094us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.245204 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.245244 (+ 40us) server_negotiation.cc:407] Connection header received
0810 22:40:49.245403 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.245410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.245666 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.245943 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:49.246688 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.248393 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.250873 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.255064 (+ 4191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.256111 (+ 1047us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.256373 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.256923 (+ 550us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.257105 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1949,"thread_start_us":378,"threads_started":1}
I20260810 22:40:49.258111 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.242672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:32889 (local address 127.0.0.1:58644)
0810 22:40:49.246253 (+ 3581us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:49.246273 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:49.246421 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:49.248842 (+ 2421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:49.248846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:49.248878 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:49.249539 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.249548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.252574 (+ 3026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.252585 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.254712 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.254736 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.257230 (+ 2494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.257237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.257440 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.257486 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 22:40:49.257643 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 22:40:49.257796 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3499,"mutex_wait_us":372,"thread_start_us":227,"threads_started":1}
I20260810 22:40:49.259229 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.242999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58644 (local address 127.19.202.66:32889)
0810 22:40:49.247086 (+ 4087us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.247092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.247254 (+ 162us) server_negotiation.cc:407] Connection header received
0810 22:40:49.247609 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.247627 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.248267 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.248612 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:49.249863 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.252260 (+ 2397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.254979 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.256550 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.256815 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.257127 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.258753 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.258951 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3879,"thread_start_us":399,"threads_started":1}
I20260810 22:40:49.259721 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.242547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.19.202.65:42855)
0810 22:40:49.244680 (+ 2133us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.244687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.244715 (+ 28us) server_negotiation.cc:407] Connection header received
0810 22:40:49.245312 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.245320 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.245666 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.246672 (+ 1006us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:49.246956 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.249633 (+ 2677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.251812 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.254505 (+ 2693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.258686 (+ 4181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.258953 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.259045 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.259179 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1982,"thread_start_us":111,"threads_started":1}
I20260810 22:40:49.262871 562 tablet_service.cc:1467] Processing CreateTablet for tablet 532491ba366e46e6b7546df61a68adf1 (DEFAULT_TABLE table=test-table [id=7378c64d0e9e467d9e3844c6a32ce593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:40:49.265008 562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532491ba366e46e6b7546df61a68adf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:49.264365 417 tablet_service.cc:1467] Processing CreateTablet for tablet 532491ba366e46e6b7546df61a68adf1 (DEFAULT_TABLE table=test-table [id=7378c64d0e9e467d9e3844c6a32ce593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:40:49.266090 417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532491ba366e46e6b7546df61a68adf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:49.276469 32734 tablet_service.cc:1467] Processing CreateTablet for tablet 532491ba366e46e6b7546df61a68adf1 (DEFAULT_TABLE table=test-table [id=7378c64d0e9e467d9e3844c6a32ce593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:40:49.278218 32734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532491ba366e46e6b7546df61a68adf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:40:49.279738 654 tablet_bootstrap.cc:492] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Bootstrap starting.
I20260810 22:40:49.280340 653 tablet_bootstrap.cc:492] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Bootstrap starting.
I20260810 22:40:49.284389 654 tablet_bootstrap.cc:654] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:49.284418 653 tablet_bootstrap.cc:654] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:49.285791 653 log.cc:824] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Log is configured to *not* fsync() on all Append() calls
I20260810 22:40:49.285791 654 log.cc:824] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Log is configured to *not* fsync() on all Append() calls
I20260810 22:40:49.290076 653 tablet_bootstrap.cc:492] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: No bootstrap required, opened a new log
I20260810 22:40:49.290407 653 ts_tablet_manager.cc:1397] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260810 22:40:49.297519 657 tablet_bootstrap.cc:492] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868: Bootstrap starting.
I20260810 22:40:49.301324 657 tablet_bootstrap.cc:654] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868: Neither blocks nor log segments found. Creating new log.
I20260810 22:40:49.302544 657 log.cc:824] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868: Log is configured to *not* fsync() on all Append() calls
I20260810 22:40:49.302865 654 tablet_bootstrap.cc:492] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: No bootstrap required, opened a new log
I20260810 22:40:49.303195 654 ts_tablet_manager.cc:1397] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.004s
I20260810 22:40:49.306481 657 tablet_bootstrap.cc:492] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868: No bootstrap required, opened a new log
I20260810 22:40:49.306771 657 ts_tablet_manager.cc:1397] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 22:40:49.309938 654 raft_consensus.cc:348] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.310456 654 raft_consensus.cc:374] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:49.310611 654 raft_consensus.cc:729] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4afde085ce4274a75fecfe19e098b9, State: Initialized, Role: FOLLOWER
I20260810 22:40:49.311384 654 consensus_queue.cc:260] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.312592 657 raft_consensus.cc:348] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.313091 657 raft_consensus.cc:374] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:49.313205 657 raft_consensus.cc:729] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8455c904fc3341c180c046ca8e9ab868, State: Initialized, Role: FOLLOWER
I20260810 22:40:49.313221 654 ts_tablet_manager.cc:1428] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260810 22:40:49.313812 634 heartbeater.cc:503] Master 127.19.202.126:38665 was elected leader, sending a full tablet report...
I20260810 22:40:49.313916 657 consensus_queue.cc:260] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.314636 653 raft_consensus.cc:348] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.315089 653 raft_consensus.cc:374] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:40:49.315205 653 raft_consensus.cc:729] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843d9b5c8bd9484aa24ae0cd7b7642aa, State: Initialized, Role: FOLLOWER
I20260810 22:40:49.315493 657 ts_tablet_manager.cc:1428] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260810 22:40:49.315891 653 consensus_queue.cc:260] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.317580 653 ts_tablet_manager.cc:1428] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Time spent starting tablet: real 0.027s user 0.009s sys 0.003s
I20260810 22:40:49.318559 488 heartbeater.cc:503] Master 127.19.202.126:38665 was elected leader, sending a full tablet report...
W20260810 22:40:49.368758 339 tablet.cc:2369] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:40:49.398249 635 tablet.cc:2369] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:40:49.438516 489 tablet.cc:2369] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:40:49.478802 662 raft_consensus.cc:482] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:40:49.479175 662 raft_consensus.cc:504] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.481441 662 leader_election.cc:288] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8455c904fc3341c180c046ca8e9ab868 (127.19.202.65:42855), 7d4afde085ce4274a75fecfe19e098b9 (127.19.202.67:45977)
I20260810 22:40:49.495456 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.483042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:44441 (local address 127.19.202.67:45977)
0810 22:40:49.483907 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.483912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.485779 (+ 1867us) server_negotiation.cc:407] Connection header received
0810 22:40:49.486174 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.486178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.486269 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.486348 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:49.489121 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.490318 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.493099 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.494066 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.494235 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.494364 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.495125 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.495243 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739}
I20260810 22:40:49.496168 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.481952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45977 (local address 127.19.202.66:44441)
0810 22:40:49.485611 (+ 3659us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:49.485639 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:49.485913 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:49.486613 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:49.486627 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:49.486657 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:49.486883 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.486901 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.490626 (+ 3725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.490630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.492831 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.492854 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.494627 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.494632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.494860 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.494909 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 22:40:49.495728 (+ 819us) client_negotiation.cc:240] Negotiation successful
0810 22:40:49.495871 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3545,"mutex_wait_us":2,"thread_start_us":2338,"threads_started":1}
I20260810 22:40:49.497325 573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532491ba366e46e6b7546df61a68adf1" candidate_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4afde085ce4274a75fecfe19e098b9" is_pre_election: true
I20260810 22:40:49.497910 573 raft_consensus.cc:2454] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843d9b5c8bd9484aa24ae0cd7b7642aa in term 0.
I20260810 22:40:49.499094 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.484127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:34835 (local address 127.19.202.65:42855)
0810 22:40:49.484342 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.484356 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.484394 (+ 38us) server_negotiation.cc:407] Connection header received
0810 22:40:49.484579 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.484583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.484713 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.484909 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:40:49.493295 (+ 8386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.494305 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.496715 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.497628 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.497805 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.497879 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.498740 (+ 861us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.498884 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 22:40:49.499099 365 leader_election.cc:302] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d4afde085ce4274a75fecfe19e098b9, 843d9b5c8bd9484aa24ae0cd7b7642aa; no voters:
I20260810 22:40:49.499743 662 raft_consensus.cc:2781] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:40:49.499855 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.481883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:42855 (local address 127.19.202.66:34835)
0810 22:40:49.483355 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:49.483383 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 22:40:49.483575 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:40:49.492560 (+ 8985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:40:49.492564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:40:49.492612 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:40:49.493030 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.493051 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.494550 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.494553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.496464 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:40:49.496486 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.498080 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.498085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:40:49.498367 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.498433 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 22:40:49.499313 (+ 880us) client_negotiation.cc:240] Negotiation successful
0810 22:40:49.499428 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1359,"mutex_wait_us":233,"thread_start_us":162,"threads_started":1}
I20260810 22:40:49.499888 662 raft_consensus.cc:482] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:40:49.499970 662 raft_consensus.cc:3037] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:49.501995 662 raft_consensus.cc:504] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.502559 662 leader_election.cc:288] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [CANDIDATE]: Term 1 election: Requested vote from peers 8455c904fc3341c180c046ca8e9ab868 (127.19.202.65:42855), 7d4afde085ce4274a75fecfe19e098b9 (127.19.202.67:45977)
I20260810 22:40:49.503645 32737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532491ba366e46e6b7546df61a68adf1" candidate_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8455c904fc3341c180c046ca8e9ab868" is_pre_election: true
I20260810 22:40:49.504091 32737 raft_consensus.cc:2454] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843d9b5c8bd9484aa24ae0cd7b7642aa in term 0.
I20260810 22:40:49.504295 573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532491ba366e46e6b7546df61a68adf1" candidate_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4afde085ce4274a75fecfe19e098b9"
I20260810 22:40:49.504480 573 raft_consensus.cc:3037] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:49.504711 32739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532491ba366e46e6b7546df61a68adf1" candidate_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8455c904fc3341c180c046ca8e9ab868"
I20260810 22:40:49.504947 32739 raft_consensus.cc:3037] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:40:49.507462 573 raft_consensus.cc:2454] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843d9b5c8bd9484aa24ae0cd7b7642aa in term 1.
I20260810 22:40:49.507652 32739 raft_consensus.cc:2454] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843d9b5c8bd9484aa24ae0cd7b7642aa in term 1.
I20260810 22:40:49.508165 365 leader_election.cc:302] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d4afde085ce4274a75fecfe19e098b9, 843d9b5c8bd9484aa24ae0cd7b7642aa; no voters:
I20260810 22:40:49.508442 662 raft_consensus.cc:2781] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:40:49.508823 662 raft_consensus.cc:686] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 1 LEADER]: Becoming Leader. State: Replica: 843d9b5c8bd9484aa24ae0cd7b7642aa, State: Running, Role: LEADER
I20260810 22:40:49.509380 662 consensus_queue.cc:237] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } }
I20260810 22:40:49.514823 32572 catalog_manager.cc:5495] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 843d9b5c8bd9484aa24ae0cd7b7642aa (127.19.202.66). New cstate: current_term: 1 leader_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843d9b5c8bd9484aa24ae0cd7b7642aa" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 32889 } health_report { overall_health: HEALTHY } } }
Aug 10 22:40:49 dist-test-slave-1x4g krb5kdc[32043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401647, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:40:49.565506 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.556742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.19.202.66:32889)
0810 22:40:49.556983 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 22:40:49.556988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:40:49.557014 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:40:49.557174 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:40:49.557177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:40:49.557285 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:40:49.557508 (+ 223us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:40:49.558220 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.559294 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.561421 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:40:49.561982 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:40:49.562153 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:40:49.562927 (+ 774us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:40:49.564165 (+ 1238us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:40:49.564364 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 22:40:49.565123 (+ 759us) server_negotiation.cc:299] Negotiation successful
0810 22:40:49.565214 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260810 22:40:49.578022 573 raft_consensus.cc:1261] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 1 FOLLOWER]: Refusing update from remote peer 843d9b5c8bd9484aa24ae0cd7b7642aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:40:49.578234 32739 raft_consensus.cc:1261] T 532491ba366e46e6b7546df61a68adf1 P 8455c904fc3341c180c046ca8e9ab868 [term 1 FOLLOWER]: Refusing update from remote peer 843d9b5c8bd9484aa24ae0cd7b7642aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:40:49.579344 662 consensus_queue.cc:1035] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8455c904fc3341c180c046ca8e9ab868" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:40:49.580160 665 consensus_queue.cc:1035] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4afde085ce4274a75fecfe19e098b9" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 22:40:49.597295 669 mvcc.cc:205] Tried to move back new op lower bound from 7317101156656578560 to 7317101156394786816. Current Snapshot: MvccSnapshot[applied={T|T < 7317101156656578560}]
I20260810 22:40:49.606289 671 mvcc.cc:205] Tried to move back new op lower bound from 7317101156656578560 to 7317101156394786816. Current Snapshot: MvccSnapshot[applied={T|T < 7317101156656578560}]
I20260810 22:40:49.615016 672 mvcc.cc:205] Tried to move back new op lower bound from 7317101156656578560 to 7317101156394786816. Current Snapshot: MvccSnapshot[applied={T|T < 7317101156656578560}]
I20260810 22:40:49.623041 32570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:40:49.623337 32570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:40:49.628018 32570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89dcadaa324e4e33be90afb3a94546b7: Sending DeleteTablet for 3 replicas of tablet 532491ba366e46e6b7546df61a68adf1
I20260810 22:40:49.630288 32734 tablet_service.cc:1514] Processing DeleteTablet for tablet 532491ba366e46e6b7546df61a68adf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:40:49 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:40992
I20260810 22:40:49.631117 562 tablet_service.cc:1514] Processing DeleteTablet for tablet 532491ba366e46e6b7546df61a68adf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:40:49 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:46730
I20260810 22:40:49.632329 417 tablet_service.cc:1514] Processing DeleteTablet for tablet 532491ba366e46e6b7546df61a68adf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:40:49 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:58644
I20260810 22:40:49.633960 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 32663
I20260810 22:40:49.650390 681 tablet_replica.cc:330] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: stopping tablet replica
I20260810 22:40:49.650859 680 tablet_replica.cc:330] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: stopping tablet replica
I20260810 22:40:49.652012 680 raft_consensus.cc:2229] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:40:49.653690 680 raft_consensus.cc:2258] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:40:49.652765 681 raft_consensus.cc:2229] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:40:49.654593 681 raft_consensus.cc:2258] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:40:49.658044 680 ts_tablet_manager.cc:1905] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:40:49.660348 681 ts_tablet_manager.cc:1905] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:40:49.664925 681 ts_tablet_manager.cc:1918] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:40:49.669122 680 ts_tablet_manager.cc:1918] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:40:49.669397 680 log.cc:1196] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/532491ba366e46e6b7546df61a68adf1
I20260810 22:40:49.670280 681 log.cc:1196] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/532491ba366e46e6b7546df61a68adf1
I20260810 22:40:49.671007 681 ts_tablet_manager.cc:1939] T 532491ba366e46e6b7546df61a68adf1 P 843d9b5c8bd9484aa24ae0cd7b7642aa: Deleting consensus metadata
I20260810 22:40:49.670105 680 ts_tablet_manager.cc:1939] T 532491ba366e46e6b7546df61a68adf1 P 7d4afde085ce4274a75fecfe19e098b9: Deleting consensus metadata
I20260810 22:40:49.673671 32553 catalog_manager.cc:4841] TS 7d4afde085ce4274a75fecfe19e098b9 (127.19.202.67:45977): tablet 532491ba366e46e6b7546df61a68adf1 (table test-table [id=7378c64d0e9e467d9e3844c6a32ce593]) successfully deleted
I20260810 22:40:49.675689 32553 catalog_manager.cc:4841] TS 843d9b5c8bd9484aa24ae0cd7b7642aa (127.19.202.66:32889): tablet 532491ba366e46e6b7546df61a68adf1 (table test-table [id=7378c64d0e9e467d9e3844c6a32ce593]) successfully deleted
W20260810 22:40:49.682071 32554 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:42855 (error 108)
I20260810 22:40:49.683310 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 346
I20260810 22:40:49.697983 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:40:49.697436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:42855 (local address 127.0.0.1:59786)
0810 22:40:49.697672 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 22:40:49.697826 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 22:40:49.698527 32554 catalog_manager.cc:4584] TS 8455c904fc3341c180c046ca8e9ab868 (127.19.202.65:42855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 532491ba366e46e6b7546df61a68adf1: Network error: Client connection negotiation failed: client connection to 127.19.202.65:42855: connect: Connection refused (error 111)
I20260810 22:40:49.704246 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 495
I20260810 22:40:49.719489 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 32535
I20260810 22:40:49.737371 20265 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 22:40:50.092588 20265 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 22:40:50.092820 20265 mini_ranger.cc:66] Stopping Ranger...
I20260810 22:40:50.447654 20265 mini_ranger.cc:68] Stopped Ranger
2026-08-10 22:40:50.448 UTC [32076] LOG: received smart shutdown request
2026-08-10 22:40:50.452 UTC [32076] LOG: background worker "logical replication launcher" (PID 32085) exited with exit code 1
2026-08-10 22:40:50.453 UTC [32080] LOG: shutting down
2026-08-10 22:40:50.623 UTC [32076] LOG: database system is shut down
2026-08-10T22:40:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49298 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:40:50 dist-test-slave-1x4g krb5kdc[693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:40:50 dist-test-slave-1x4g krb5kdc[693](info): set up 2 sockets
Aug 10 22:40:50 dist-test-slave-1x4g krb5kdc[693](info): commencing operation
krb5kdc: starting...
W20260810 22:40:51.837598 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:40:51 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:40:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:40:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/rangeradmin_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/rangeradmin_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/rangerlookup_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/rangerlookup_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
I20260810 22:40:52.005496 20265 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskx9HXXp/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/postgres -l logfile start
2026-08-10 22:40:54.412 UTC [724] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 22:40:54.412 UTC [724] LOG: listening on IPv4 address "127.19.202.84", port 40065
2026-08-10 22:40:54.416 UTC [724] LOG: listening on Unix socket "/tmp/.s.PGSQL.40065"
2026-08-10 22:40:54.451 UTC [727] LOG: database system was shut down at 2026-08-10 22:40:53 UTC
2026-08-10 22:40:54.455 UTC [724] LOG: database system is ready to accept connections
I20260810 22:40:55.431187 20265 mini_postgres.cc:96] Postgres bound to 40065
2026-08-10 22:40:55.439 UTC [738] FATAL: database "slave" does not exist
127.19.202.84:40065 - accepting connections
I20260810 22:40:55.439647 20265 mini_ranger.cc:161] Starting Ranger...
I20260810 22:40:55.456271 20265 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 22:40:55.891999 20265 mini_ranger.cc:87] Created ranger Postgres database
I20260810 22:40:55.892091 20265 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-admin
2026-08-10 22:40:56,169 [I] DB FLAVOR :POSTGRES
2026-08-10 22:40:56,169 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 22:40:56,169 [I] Checking connection..
2026-08-10 22:40:56,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 22:40:56,419 [I] Checking connection passed.
2026-08-10 22:40:56,419 [I] --------- Verifying version history table ---------
2026-08-10 22:40:56,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:40:56,660 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 22:40:56,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:40:56,906 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 22:40:56,906 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 22:40:56,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 22:40:57.149 UTC [807] WARNING: there is no transaction in progress
2026-08-10 22:40:57,155 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 22:40:57,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:40:57,406 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 22:40:57,406 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 22:40:57,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 22:40:57,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 22:40:57,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-10 22:40:58,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 22:40:58,392 [I] Table x_portal_user does not exist in database ranger
2026-08-10 22:40:58,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 22:40:58,632 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 22:40:58,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 22:40:58,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 22:40:59,134 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 22:40:59,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 22:41:00.023 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.039 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.054 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.068 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.083 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.102 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.165 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.171 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.185 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.194 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.204 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.216 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.225 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.231 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.238 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00.244 UTC [927] WARNING: there is no transaction in progress
2026-08-10 22:41:00,860 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 22:41:00,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 22:41:01,092 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 22:41:01,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 22:41:01,354 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 22:41:01,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 22:41:01,605 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 22:41:01,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 22:41:01,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 22:41:02,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-10 22:41:02,319 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 22:41:02,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 22:41:02,587 [I] DB_PATCHES have already been applied
I20260810 22:41:02.594002 20265 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-admin:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskx9HXXp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 22:41:02.594072 20265 mini_ranger.cc:193] Using host: 127.19.202.84
I20260810 22:41:02.598901 20265 mini_ranger.cc:227] Ranger admin URL: http://127.19.202.84:40735
Aug 10, 2026 10:41:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 10:41:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 10:41:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 10:41:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 10:41:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 10:41:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 10:41:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 22:41:03.839159 20265 mini_ranger.cc:160] Time spent starting Ranger: real 8.400s user 0.000s sys 0.012s
Aug 10, 2026 10:41:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 10:41:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 10:41:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 10:41:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 10:41:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 22:41:24.378867 20265 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.202.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.202.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/rangerkms_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.202.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/rangerkms_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.202.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.202.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.202.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/HTTP_127.19.202.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 22:41:24 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401684, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 22:41:24.470604 20265 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 22:41:24.480485 20265 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 22:41:25.336097 20265 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 22:41:25.336197 20265 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms
I20260810 22:41:25.336220 20265 mini_ranger_kms.cc:227] Using postgres at 127.19.202.84:40065
2026-08-10 22:41:25,578 [I] DB FLAVOR :POSTGRES
2026-08-10 22:41:25,578 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 22:41:25,578 [I] Checking connection
2026-08-10 22:41:25,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 22:41:25,837 [I] connection success
2026-08-10 22:41:25,838 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 22:41:25,838 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 22:41:25,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 22:41:26,102 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 22:41:26,102 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 22:41:26,102 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 22:41:26,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.202.84:40065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 22:41:26,379 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 22:41:26.792049 20265 mini_ranger_kms.cc:310] Created kms service
I20260810 22:41:26.975366 20265 mini_ranger_kms.cc:326] Created kudu user
I20260810 22:41:27.021879 20265 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 22:41:27 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401684, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:27.406847 20265 mini_ranger_kms.cc:384] Added ranger policy
I20260810 22:41:27.407141 20265 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms:/tmp/dist-test-taskx9HXXp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskx9HXXp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 22:41:27.407177 20265 mini_ranger_kms.cc:240] Using host: 127.19.202.84
I20260810 22:41:27.411767 20265 mini_ranger_kms.cc:276] Ranger KMS PID: 1128
I20260810 22:41:27.411847 20265 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.202.84:55925
Aug 10, 2026 10:41:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 10:41:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 10:41:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 10:41:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 10:41:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 10:41:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 22:41:29 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786401689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.202.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:29.771929 20265 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.301s user 0.014s sys 0.201s
I20260810 22:41:29.772095 20265 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 22:41:29.772176 20265 mini_ranger_kms.cc:401] 127.19.202.84:55925/kms/v1/keys
Aug 10 22:41:30 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.202.84@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
Aug 10 22:41:30 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.202.84@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
Aug 10 22:41:32 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:41:32.719838 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34331
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34331
--ranger_config_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:32.829879 1181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:32.830130 1181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:32.830201 1181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:32.835132 1181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:32.835258 1181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:32.835297 1181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:32.835330 1181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:32.835361 1181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:32.841071 1181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.202.84:55925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34331
--ranger_config_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:32.842002 1181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:32.843001 1181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:32.851164 1186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:32.851171 1187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:32.851191 1189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:32.851248 1181 server_base.cc:1034] running on GCE node
I20260810 22:41:32.851931 1181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:32.852967 1181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:32.854135 1181 hybrid_clock.cc:648] HybridClock initialized: now 1786401692854119 us; error 29 us; skew 500 ppm
Aug 10 22:41:32 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401692, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:32.856839 1181 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:41:32.857709 1181 webserver.cc:460] Webserver started at http://127.19.202.126:39285/ using document root <none> and password file <none>
I20260810 22:41:32.858275 1181 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:32.858356 1181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:32.858556 1181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:41:32 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401692, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:32.911486 1181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "6071d82b64504c71adb3d476e115b283"
format_stamp: "Formatted at 2026-08-10 22:41:32 on dist-test-slave-1x4g"
server_key: "5538a6fb30e70926a2c0473492398fbd"
server_key_iv: "a938bf8d95f2851d41a6bff12cb8afb8"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:32.912128 1181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "6071d82b64504c71adb3d476e115b283"
format_stamp: "Formatted at 2026-08-10 22:41:32 on dist-test-slave-1x4g"
server_key: "5538a6fb30e70926a2c0473492398fbd"
server_key_iv: "a938bf8d95f2851d41a6bff12cb8afb8"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:32.916517 1181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 22:41:32.931232 1196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:32.932636 1181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:41:32.932775 1181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "6071d82b64504c71adb3d476e115b283"
format_stamp: "Formatted at 2026-08-10 22:41:32 on dist-test-slave-1x4g"
server_key: "5538a6fb30e70926a2c0473492398fbd"
server_key_iv: "a938bf8d95f2851d41a6bff12cb8afb8"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:32.932948 1181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:32.943367 1181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:32.947307 1181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:32.947489 1181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:32.955363 1181 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:34331
I20260810 22:41:32.955367 1252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:34331 every 8 connection(s)
I20260810 22:41:32.956343 1181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:41:32.959102 1253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:32.964504 1253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Bootstrap starting.
I20260810 22:41:32.966547 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 1181
I20260810 22:41:32.966670 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:41:32.966974 20265 external_mini_cluster.cc:1428] Setting key 7f128cd11acd230c88ea6d1eb813a597
I20260810 22:41:32.967075 1253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:32.967962 1253 log.cc:824] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:32.970386 1253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: No bootstrap required, opened a new log
Aug 10 22:41:32 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:32.975145 1253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:32.975436 1253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:32.975543 1253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6071d82b64504c71adb3d476e115b283, State: Initialized, Role: FOLLOWER
I20260810 22:41:32.976097 1253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:32.976265 1253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:41:32.976356 1253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:41:32.976466 1253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:32.977589 1253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:32.977972 1253 leader_election.cc:302] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6071d82b64504c71adb3d476e115b283; no voters:
I20260810 22:41:32.978344 1253 leader_election.cc:288] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:41:32.978452 1258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:41:32.979607 1253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:32.978986 1258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 LEADER]: Becoming Leader. State: Replica: 6071d82b64504c71adb3d476e115b283, State: Running, Role: LEADER
I20260810 22:41:32.981057 1258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:32.981185 1256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:32.968181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50638 (local address 127.19.202.126:34331)
0810 22:41:32.968653 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:32.968674 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:32.968717 (+ 43us) server_negotiation.cc:407] Connection header received
0810 22:41:32.969410 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:32.969427 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:32.969712 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:32.970038 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:32.970896 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:32.971872 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:32.972817 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:32.973297 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:32.973480 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:32.976070 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:32.976075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:32.976088 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:32.976113 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:32.978000 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:32.978627 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:32.978634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:32.978641 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:32.978760 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:32.979062 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:32.979066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:32.979068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:32.979450 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:32.979660 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:32.979920 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 22:41:32.980212 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":132,"threads_started":1}
I20260810 22:41:32.982628 1259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6071d82b64504c71adb3d476e115b283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } } }
I20260810 22:41:32.982762 1259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:32.982923 1260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6071d82b64504c71adb3d476e115b283. Latest consensus state: current_term: 1 leader_uuid: "6071d82b64504c71adb3d476e115b283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } } }
I20260810 22:41:32.982669 1253 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 22:41:32.983003 1260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:32.983492 1272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:41:32.990967 1272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:41:33.002769 1272 catalog_manager.cc:1347] Generated new cluster ID: ccefb3e76498445aa9e2d5656a5a12c3
I20260810 22:41:33.002843 1272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:41:33.037634 1272 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:41:33.038928 1272 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:41:33.046546 1272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Generated new TSK 0
I20260810 22:41:33.047314 1272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:41:33.693118 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34331
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:33.812063 1300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:33.812387 1300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:33.812453 1300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:33.816527 1300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:33.816597 1300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:33.816654 1300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:41:33.822106 1300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.202.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:33.823174 1300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:33.824242 1300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:33.836269 1311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:33.836681 1309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:33.837306 1308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:33.837906 1300 server_base.cc:1034] running on GCE node
I20260810 22:41:33.838325 1300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:33.838953 1300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:33.840126 1300 hybrid_clock.cc:648] HybridClock initialized: now 1786401693840095 us; error 43 us; skew 500 ppm
Aug 10 22:41:33 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401693, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:33.843246 1300 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:41:33.844282 1300 webserver.cc:460] Webserver started at http://127.19.202.65:43175/ using document root <none> and password file <none>
I20260810 22:41:33.844936 1300 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:33.845032 1300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:33.845264 1300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:41:33 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401693, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:33.859094 1300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "763cc216a510462c96afa6511d47448b"
format_stamp: "Formatted at 2026-08-10 22:41:33 on dist-test-slave-1x4g"
server_key: "b754d1c34ae35c0db918ceeba4839a6b"
server_key_iv: "5742b235f3432f66db94dd76268e6c9d"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:33.859833 1300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "763cc216a510462c96afa6511d47448b"
format_stamp: "Formatted at 2026-08-10 22:41:33 on dist-test-slave-1x4g"
server_key: "b754d1c34ae35c0db918ceeba4839a6b"
server_key_iv: "5742b235f3432f66db94dd76268e6c9d"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:33.865203 1300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 22:41:33.883162 1320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:33.884838 1300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:41:33.884975 1300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "763cc216a510462c96afa6511d47448b"
format_stamp: "Formatted at 2026-08-10 22:41:33 on dist-test-slave-1x4g"
server_key: "b754d1c34ae35c0db918ceeba4839a6b"
server_key_iv: "5742b235f3432f66db94dd76268e6c9d"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:33.885104 1300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:33.910499 1300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:33.914059 1300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:33.914301 1300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:33.915067 1300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:33.916107 1300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:33.916219 1300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:33.916349 1300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:33.916435 1300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:33.930073 1300 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:42439
I20260810 22:41:33.931177 1300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:41:33.932710 1448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:42439 every 8 connection(s)
I20260810 22:41:33.941093 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 1300
I20260810 22:41:33.941241 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:41:33.941552 20265 external_mini_cluster.cc:1428] Setting key 9d7efbe960c976279332e4c18ea9b041
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
Aug 10 22:41:33 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401693, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:33.963577 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:33.939177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34331 (local address 127.19.202.65:48353)
0810 22:41:33.947156 (+ 7979us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:33.947204 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:33.948063 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:33.948837 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:33.948850 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:33.949232 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:33.950012 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:33.950036 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:33.952466 (+ 2430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:33.952472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:33.954151 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:33.954166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:33.954950 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:33.954955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:33.955176 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:33.955900 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:33.955927 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:33.957821 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:33.960170 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:33.960177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:33.960193 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:33.960548 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:33.960924 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:33.960927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:33.960929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:33.961108 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:33.961708 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:33.961715 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:33.962036 (+ 321us) client_negotiation.cc:769] Sending connection context
0810 22:41:33.962217 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 22:41:33.962534 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7721,"thread_start_us":139,"threads_started":1}
I20260810 22:41:33.964959 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:33.940302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:48353 (local address 127.19.202.126:34331)
0810 22:41:33.948418 (+ 8116us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:33.948424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:33.948434 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:41:33.948467 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:33.948469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:33.948504 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:33.948609 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:33.951226 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:33.952193 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:33.954326 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:33.954736 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:33.957420 (+ 2684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:33.958030 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:33.958034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:33.958037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:33.958064 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:33.960033 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:33.960675 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:33.960679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:33.960681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:33.960774 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:33.961226 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:33.961229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:33.961231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:33.961574 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:33.964527 (+ 2953us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:33.964604 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 22:41:33.964723 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8037,"thread_start_us":93,"threads_started":1}
I20260810 22:41:33.966295 1449 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34331
I20260810 22:41:33.966547 1449 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:33.967136 1449 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:33.969034 1212 ts_manager.cc:194] Registered new tserver with Master: 763cc216a510462c96afa6511d47448b (127.19.202.65:42439)
I20260810 22:41:33.970731 1212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:48353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:41:33.992022 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34331
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 22:41:34.144673 1457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:34.145025 1457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:34.145088 1457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:34.150782 1457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:34.150853 1457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:34.150933 1457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:41:34.158036 1457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.202.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:34.159336 1457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:34.160517 1457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:34.172039 1465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:34.172297 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:34.172466 1467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:34.173045 1457 server_base.cc:1034] running on GCE node
I20260810 22:41:34.173513 1457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:34.174249 1457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:34.175443 1457 hybrid_clock.cc:648] HybridClock initialized: now 1786401694175381 us; error 78 us; skew 500 ppm
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:34.179615 1457 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:41:34.180624 1457 webserver.cc:460] Webserver started at http://127.19.202.66:33977/ using document root <none> and password file <none>
I20260810 22:41:34.181277 1457 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:34.181388 1457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:34.181708 1457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:34.197134 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "9e3fe10a224c4772bb53831f3c00234e"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "1b4230637e4fd79bd3ad0a818f09cc55"
server_key_iv: "6138a39d77128018e4e30c9547880b70"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:34.197860 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "9e3fe10a224c4772bb53831f3c00234e"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "1b4230637e4fd79bd3ad0a818f09cc55"
server_key_iv: "6138a39d77128018e4e30c9547880b70"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:34.203114 1457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 22:41:34.230508 1474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:34.242617 1457 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.004s sys 0.001s
I20260810 22:41:34.242858 1457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "9e3fe10a224c4772bb53831f3c00234e"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "1b4230637e4fd79bd3ad0a818f09cc55"
server_key_iv: "6138a39d77128018e4e30c9547880b70"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:34.243193 1457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:34.262792 1457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:34.266111 1457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:34.266305 1457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:34.266950 1457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:34.267840 1457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:34.267917 1457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:34.268013 1457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:34.268064 1457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:34.279991 1457 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:41667
I20260810 22:41:34.281057 1457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:41:34.285167 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 1457
I20260810 22:41:34.285300 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:41:34.285578 20265 external_mini_cluster.cc:1428] Setting key 31681a495465fdb1f98720aba523e67f
I20260810 22:41:34.289288 1600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:41667 every 8 connection(s)
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:41:34.323431 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:34.284236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:50775 (local address 127.19.202.126:34331)
0810 22:41:34.284360 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:34.284364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:34.284380 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:41:34.286040 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:34.286043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:34.286106 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:34.286161 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:34.305720 (+ 19559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.306519 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.308601 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.308986 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.309071 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:34.313528 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:34.313532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:34.313535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:34.313564 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:34.315384 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:34.316021 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:34.316024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:34.316026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:34.316111 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:34.316515 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:34.316519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:34.316521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:34.316785 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:34.316891 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:34.323047 (+ 6156us) server_negotiation.cc:299] Negotiation successful
0810 22:41:34.323207 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:41:34.325492 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:34.283255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34331 (local address 127.19.202.66:50775)
0810 22:41:34.283729 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:34.283783 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:34.284695 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:34.286579 (+ 1884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:34.286587 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:34.286920 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:34.305526 (+ 18606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:34.305545 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.306703 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.306707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:34.307998 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:34.308011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.309291 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.309295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:34.309522 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:34.310284 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:34.310305 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:34.313303 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:34.315533 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:34.315539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:34.315556 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:34.315903 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:34.316214 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:34.316216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:34.316218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:34.316391 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:34.316961 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:34.316968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:34.317298 (+ 330us) client_negotiation.cc:769] Sending connection context
0810 22:41:34.323906 (+ 6608us) client_negotiation.cc:240] Negotiation successful
0810 22:41:34.324323 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":135,"threads_started":1}
I20260810 22:41:34.327042 1601 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34331
I20260810 22:41:34.327279 1601 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:34.327859 1601 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:34.329085 1212 ts_manager.cc:194] Registered new tserver with Master: 9e3fe10a224c4772bb53831f3c00234e (127.19.202.66:41667)
I20260810 22:41:34.329787 1212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:50775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:41:34.364032 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34331
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:34.544557 1606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:34.545078 1606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:34.545202 1606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
W20260810 22:41:34.551638 1606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:34.551769 1606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:34.551899 1606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:41:34.560129 1606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.202.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:34.563458 1606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:34.565747 1606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:41:34.587558 1606 server_base.cc:1034] running on GCE node
W20260810 22:41:34.588088 1611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:34.588413 1614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:34.588889 1612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:34.589710 1606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:34.590678 1606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:34.595546 1606 hybrid_clock.cc:648] HybridClock initialized: now 1786401694595455 us; error 98 us; skew 500 ppm
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:34.599852 1606 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:41:34.600952 1606 webserver.cc:460] Webserver started at http://127.19.202.67:40505/ using document root <none> and password file <none>
I20260810 22:41:34.601720 1606 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:34.601861 1606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:34.602130 1606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:34.622669 1606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "14a558354a504f5eadb48bed37964f89"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "0c1026cc48a80592a793ec8cdf199e03"
server_key_iv: "04d562fbb1f57e3d57baecdebdf8d5e9"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:34.625554 1606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "14a558354a504f5eadb48bed37964f89"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "0c1026cc48a80592a793ec8cdf199e03"
server_key_iv: "04d562fbb1f57e3d57baecdebdf8d5e9"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:34.650861 1606 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.004s sys 0.000s
I20260810 22:41:34.678440 1621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:34.680675 1606 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260810 22:41:34.680917 1606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "14a558354a504f5eadb48bed37964f89"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "0c1026cc48a80592a793ec8cdf199e03"
server_key_iv: "04d562fbb1f57e3d57baecdebdf8d5e9"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:34.681062 1606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:34.714960 1606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:34.718859 1606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:34.719127 1606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:34.719875 1606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:34.720950 1606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:34.721069 1606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:34.721185 1606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:34.721246 1606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:34.753585 1606 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:38933
I20260810 22:41:34.754878 1606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:41:34.756883 1746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:38933 every 8 connection(s)
I20260810 22:41:34.763880 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 1606
I20260810 22:41:34.764002 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:41:34.764297 20265 external_mini_cluster.cc:1428] Setting key 263a0ce662822fb88db9c6a6f533b429
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:34.785506 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:34.759219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:59323 (local address 127.19.202.126:34331)
0810 22:41:34.760698 (+ 1479us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:34.760703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:34.768487 (+ 7784us) server_negotiation.cc:407] Connection header received
0810 22:41:34.769837 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:34.769840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:34.769920 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:34.769979 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:34.771973 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.772768 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.774501 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.774828 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.774908 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:34.779180 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:34.779183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:34.779186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:34.779210 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:34.782412 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:34.783303 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:34.783307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:34.783309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:34.783402 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:34.783973 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:34.783976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:34.783978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:34.784226 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:34.784353 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:34.785225 (+ 872us) server_negotiation.cc:299] Negotiation successful
0810 22:41:34.785348 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376}
I20260810 22:41:34.787178 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:34.759585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34331 (local address 127.19.202.67:59323)
0810 22:41:34.768345 (+ 8760us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:34.768404 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:34.769590 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:34.770589 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:34.770598 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:34.771061 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:34.771796 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:34.771816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.772988 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.772991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:34.774362 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:34.774373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:34.775100 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:34.775103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:34.775339 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:34.776086 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:34.776106 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:34.779016 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:34.782573 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:34.782579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:34.782603 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:34.783142 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:34.783531 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:34.783535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:34.783537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:34.783844 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:34.784392 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:34.784399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:34.784898 (+ 499us) client_negotiation.cc:769] Sending connection context
0810 22:41:34.785166 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 22:41:34.785637 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7487,"thread_start_us":225,"threads_started":1}
I20260810 22:41:34.796940 1747 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34331
I20260810 22:41:34.797276 1747 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:34.797914 1747 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:34.799041 1212 ts_manager.cc:194] Registered new tserver with Master: 14a558354a504f5eadb48bed37964f89 (127.19.202.67:38933)
I20260810 22:41:34.799759 1212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:59323
I20260810 22:41:34.803596 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:41:34.804324 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 1300
I20260810 22:41:34.815698 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 1457
I20260810 22:41:34.825114 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 1606
I20260810 22:41:34.838129 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 1181
I20260810 22:41:34.848193 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34331
--webserver_interface=127.19.202.126
--webserver_port=39285
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34331
--ranger_config_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:34.957284 1750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:34.957525 1750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:34.957597 1750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:34.961534 1750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:34.961586 1750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:34.961606 1750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:34.961622 1750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:34.961637 1750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:34.966917 1750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34331
--ranger_config_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=39285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:34.967892 1750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:34.968935 1750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:34.977667 1756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:34.977707 1758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:34.977726 1755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:34.977869 1750 server_base.cc:1034] running on GCE node
I20260810 22:41:34.978588 1750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:34.979678 1750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:34.980842 1750 hybrid_clock.cc:648] HybridClock initialized: now 1786401694980820 us; error 33 us; skew 500 ppm
Aug 10 22:41:34 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:34.983587 1750 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:41:34.984447 1750 webserver.cc:460] Webserver started at http://127.19.202.126:39285/ using document root <none> and password file <none>
I20260810 22:41:34.985049 1750 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:34.985142 1750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401694, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:35.227303 1750 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:41:35.229764 1767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:35.231344 1750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 22:41:35.231523 1750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "6071d82b64504c71adb3d476e115b283"
format_stamp: "Formatted at 2026-08-10 22:41:32 on dist-test-slave-1x4g"
server_key: "5538a6fb30e70926a2c0473492398fbd"
server_key_iv: "a938bf8d95f2851d41a6bff12cb8afb8"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:35.232120 1750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:35.247624 1750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:35.250608 1750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:35.250808 1750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:35.257859 1750 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:34331
I20260810 22:41:35.257889 1828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:34331 every 8 connection(s)
I20260810 22:41:35.258869 1750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:41:35.263160 1829 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:41:35.264948 1829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Bootstrap starting.
I20260810 22:41:35.268877 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 1750
I20260810 22:41:35.269317 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:42439
--local_ip_for_outbound_sockets=127.19.202.65
--tserver_master_addrs=127.19.202.126:34331
--webserver_port=43175
--webserver_interface=127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:35.304380 1829 log.cc:824] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:35.310895 1829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:41:35.311348 1829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Bootstrap complete.
I20260810 22:41:35.315471 1829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:35.315863 1829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6071d82b64504c71adb3d476e115b283, State: Initialized, Role: FOLLOWER
I20260810 22:41:35.316324 1829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:35.316473 1829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:41:35.316555 1829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:41:35.316699 1829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:41:35.318514 1829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:35.318840 1829 leader_election.cc:302] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6071d82b64504c71adb3d476e115b283; no voters:
I20260810 22:41:35.319185 1829 leader_election.cc:288] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 22:41:35.319381 1833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 22:41:35.319696 1833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [term 2 LEADER]: Becoming Leader. State: Replica: 6071d82b64504c71adb3d476e115b283, State: Running, Role: LEADER
I20260810 22:41:35.320081 1833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } }
I20260810 22:41:35.320349 1829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:35.321622 1834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6071d82b64504c71adb3d476e115b283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } } }
I20260810 22:41:35.321751 1834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:35.322139 1842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:41:35.322103 1835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6071d82b64504c71adb3d476e115b283. Latest consensus state: current_term: 2 leader_uuid: "6071d82b64504c71adb3d476e115b283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6071d82b64504c71adb3d476e115b283" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34331 } } }
I20260810 22:41:35.322273 1835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:35.325438 1842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:41:35.325980 1829 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 22:41:35.326068 1842 catalog_manager.cc:1259] Loaded cluster ID: ccefb3e76498445aa9e2d5656a5a12c3
I20260810 22:41:35.326122 1842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:41:35.328552 1842 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:41:35.329244 1842 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Loaded TSK: 0
I20260810 22:41:35.330484 1842 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 22:41:35.418946 1831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:35.419262 1831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:35.419330 1831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:35.425213 1831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:35.425292 1831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:35.425396 1831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:41:35.432713 1831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:42439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=43175
--tserver_master_addrs=127.19.202.126:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:35.433917 1831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:35.435118 1831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:35.447031 1869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:35.447615 1870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:35.448055 1831 server_base.cc:1034] running on GCE node
W20260810 22:41:35.448397 1872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:35.448870 1831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:35.449527 1831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:35.450728 1831 hybrid_clock.cc:648] HybridClock initialized: now 1786401695450659 us; error 85 us; skew 500 ppm
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:35.454421 1831 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:41:35.455549 1831 webserver.cc:460] Webserver started at http://127.19.202.65:43175/ using document root <none> and password file <none>
I20260810 22:41:35.456368 1831 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:35.456509 1831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:35.490638 1831 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260810 22:41:35.496731 1881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:35.498430 1831 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260810 22:41:35.498562 1831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "763cc216a510462c96afa6511d47448b"
format_stamp: "Formatted at 2026-08-10 22:41:33 on dist-test-slave-1x4g"
server_key: "b754d1c34ae35c0db918ceeba4839a6b"
server_key_iv: "5742b235f3432f66db94dd76268e6c9d"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:35.499264 1831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:35.523001 1831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:35.526513 1831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:35.526727 1831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:35.527359 1831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:35.528311 1831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:35.528410 1831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:35.528522 1831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:35.528589 1831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:35.538407 1831 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:42439
I20260810 22:41:35.539577 1831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:41:35.542645 2002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:42439 every 8 connection(s)
I20260810 22:41:35.547614 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 1831
I20260810 22:41:35.548050 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:41667
--local_ip_for_outbound_sockets=127.19.202.66
--tserver_master_addrs=127.19.202.126:34331
--webserver_port=33977
--webserver_interface=127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:35.565783 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:35.544659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34331 (local address 127.19.202.65:49055)
0810 22:41:35.545391 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:35.545441 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:35.546203 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:35.547377 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:35.547385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:35.547733 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:35.548269 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:35.548286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.549949 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.549952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:35.551173 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:35.551183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.552289 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.552294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:35.552505 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:35.556937 (+ 4432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:35.556959 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:35.558713 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:35.562348 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:35.562355 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:35.562367 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:35.562685 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:35.563124 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:35.563127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:35.563129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:35.563293 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:35.564002 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:35.564008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:35.564302 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 22:41:35.564470 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 22:41:35.564812 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":159,"threads_started":1}
I20260810 22:41:35.568157 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:35.544497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:49055 (local address 127.19.202.126:34331)
0810 22:41:35.545122 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:35.545144 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:35.546457 (+ 1313us) server_negotiation.cc:407] Connection header received
0810 22:41:35.546698 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:35.546707 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:35.546992 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:35.547297 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:35.548450 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.549777 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.551361 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.552100 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.552274 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:35.558929 (+ 6655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:35.558936 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:35.558950 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:35.558981 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:35.562181 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:35.562836 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:35.562842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:35.562848 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:35.562991 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:35.563422 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:35.563425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:35.563427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:35.563870 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:35.566834 (+ 2964us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:35.566981 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 22:41:35.567260 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":112,"threads_started":1}
I20260810 22:41:35.571111 2003 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34331
I20260810 22:41:35.571359 2003 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:35.571940 2003 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
W20260810 22:41:35.574246 2003 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:34331 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 22:41:35.574379 2003 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:35.574549 2003 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:35.575495 2003 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:35.575673 2003 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
W20260810 22:41:35.576524 2003 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:41:35.692505 2007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:35.692839 2007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:35.692931 2007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:35.697387 2007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:35.697458 2007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:35.697541 2007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:41:35.703387 2007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:41667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=33977
--tserver_master_addrs=127.19.202.126:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:35.704386 2007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:35.705454 2007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:35.716749 2015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:35.717226 2013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:35.717700 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:35.718745 2007 server_base.cc:1034] running on GCE node
I20260810 22:41:35.719167 2007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:35.719810 2007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:35.721027 2007 hybrid_clock.cc:648] HybridClock initialized: now 1786401695720995 us; error 43 us; skew 500 ppm
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:35.727249 2007 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:41:35.728370 2007 webserver.cc:460] Webserver started at http://127.19.202.66:33977/ using document root <none> and password file <none>
I20260810 22:41:35.729151 2007 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:35.729305 2007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:35.750167 2007 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 22:41:35.753628 2025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:35.755138 2007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 22:41:35.755257 2007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "9e3fe10a224c4772bb53831f3c00234e"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "1b4230637e4fd79bd3ad0a818f09cc55"
server_key_iv: "6138a39d77128018e4e30c9547880b70"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:35.755936 2007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:35.783061 2007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:35.786656 2007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:35.786846 2007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:35.787500 2007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:35.788321 2007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:35.788388 2007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:35.788452 2007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:35.788483 2007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:35.802191 2007 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:41667
I20260810 22:41:35.802433 2148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:41667 every 8 connection(s)
I20260810 22:41:35.803733 2007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:41:35.809605 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2007
I20260810 22:41:35.810106 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.202.84:55925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:38933
--local_ip_for_outbound_sockets=127.19.202.67
--tserver_master_addrs=127.19.202.126:34331
--webserver_port=40505
--webserver_interface=127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:35.821439 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:35.806114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:52773 (local address 127.19.202.126:34331)
0810 22:41:35.806275 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:35.806280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:35.807221 (+ 941us) server_negotiation.cc:407] Connection header received
0810 22:41:35.808450 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:35.808453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:35.808503 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:35.808565 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:35.810091 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.810902 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.812702 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.813143 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.813263 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:35.816260 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:35.816267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:35.816273 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:35.816307 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:35.818361 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:35.819105 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:35.819111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:35.819113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:35.819211 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:35.819668 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:35.819674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:35.819677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:35.819988 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:35.820079 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:35.820978 (+ 899us) server_negotiation.cc:299] Negotiation successful
0810 22:41:35.821180 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 22:41:35.823730 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:35.806543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34331 (local address 127.19.202.66:52773)
0810 22:41:35.807041 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:35.807095 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:35.808188 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:35.808813 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:35.808823 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:35.809162 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:35.809915 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:35.809938 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.811062 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.811066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:35.812557 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:35.812568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:35.813456 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:35.813460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:35.813691 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:35.814398 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:35.814424 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:35.816043 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:35.818525 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:35.818535 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:35.818552 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:35.818970 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:35.819339 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:35.819342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:35.819344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:35.819539 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:35.820196 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:35.820205 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:35.820607 (+ 402us) client_negotiation.cc:769] Sending connection context
0810 22:41:35.822109 (+ 1502us) client_negotiation.cc:240] Negotiation successful
0810 22:41:35.822491 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":114,"threads_started":1}
I20260810 22:41:35.825577 2149 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34331
I20260810 22:41:35.825845 2149 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:35.826403 2149 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
W20260810 22:41:35.828037 2149 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:34331 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 22:41:35.828150 2149 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:35.828295 2149 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:35.829131 2149 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:35.829304 2149 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
W20260810 22:41:35.830019 2149 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:41:35.945204 2152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:35.945617 2152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:35.945726 2152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:35.951758 2152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:35.951884 2152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:35.952008 2152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:41:35.960022 2152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.202.84:55925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:38933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=40505
--tserver_master_addrs=127.19.202.126:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:35.961465 2152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:35.962742 2152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:35.976715 2158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:35.976879 2160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:35.976845 2152 server_base.cc:1034] running on GCE node
W20260810 22:41:35.976717 2157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:35.977811 2152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:35.978632 2152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:35.981954 2152 hybrid_clock.cc:648] HybridClock initialized: now 1786401695981923 us; error 43 us; skew 500 ppm
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:35.985873 2152 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:41:35.987085 2152 webserver.cc:460] Webserver started at http://127.19.202.67:40505/ using document root <none> and password file <none>
I20260810 22:41:35.987831 2152 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:35.987950 2152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 22:41:35 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:36.011399 2152 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 22:41:36.014626 2168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:36.016254 2152 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 22:41:36.016389 2152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "14a558354a504f5eadb48bed37964f89"
format_stamp: "Formatted at 2026-08-10 22:41:34 on dist-test-slave-1x4g"
server_key: "0c1026cc48a80592a793ec8cdf199e03"
server_key_iv: "04d562fbb1f57e3d57baecdebdf8d5e9"
server_key_version: "kuduclusterkey@0"
I20260810 22:41:36.017197 2152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:36.050043 2152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:36.053493 2152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:36.053701 2152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:36.054297 2152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:36.055176 2152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:36.055248 2152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:36.055312 2152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:36.055346 2152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:36.066051 2152 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:38933
I20260810 22:41:36.066075 2294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:38933 every 8 connection(s)
I20260810 22:41:36.067332 2152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:41:36.068650 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2152
I20260810 22:41:36.079861 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.069869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50648 (local address 127.19.202.126:34331)
0810 22:41:36.070002 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:36.070007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:36.070021 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:41:36.070068 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:36.070073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:36.070127 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:36.070210 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:36.071694 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.072526 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.073865 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.074217 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.074315 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.076213 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:36.076218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:36.076220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:36.076246 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:36.078114 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:36.078570 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:36.078576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:36.078578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:36.078665 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:36.078957 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:36.078961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:36.078963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:36.079148 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:36.079338 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:36.079564 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 22:41:36.079684 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 22:41:36 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401695, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:36.088418 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.071714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:38025 (local address 127.19.202.126:34331)
0810 22:41:36.077486 (+ 5772us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:36.077507 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:36.077527 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:41:36.077590 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:36.077594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:36.077636 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:36.079176 (+ 1540us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:36.079279 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.080083 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.081789 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.082149 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.082254 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.084547 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:36.084550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:36.084553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:36.084577 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:36.086209 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:36.086708 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:36.086711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:36.086713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:36.086805 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:36.087287 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:36.087292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:36.087294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:36.087517 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:36.087616 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:36.088098 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 22:41:36.088220 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5456,"thread_start_us":97,"threads_started":1}
I20260810 22:41:36.089221 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.070567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34331 (local address 127.19.202.67:38025)
0810 22:41:36.075260 (+ 4693us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:36.075322 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:36.076460 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:36.077963 (+ 1503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:36.077975 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:36.078322 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:36.079026 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.079049 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.080267 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.080271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.081674 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.081683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.082296 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.082298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.082452 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.082993 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:36.083008 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:36.084377 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:36.086343 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:36.086353 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:36.086365 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:36.086598 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:36.087034 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:36.087037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:36.087038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:36.087177 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:36.087618 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:36.087623 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:36.087844 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 22:41:36.088022 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 22:41:36.088260 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":133,"threads_started":1}
I20260810 22:41:36.090528 2295 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34331
I20260810 22:41:36.090765 2295 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:36.091261 2295 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:36.092669 1789 ts_manager.cc:194] Registered new tserver with Master: 14a558354a504f5eadb48bed37964f89 (127.19.202.67:38933)
I20260810 22:41:36.094039 1789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:38025
Aug 10 22:41:36 dist-test-slave-1x4g krb5kdc[693](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786401696, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:36.576786 2003 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:36.577147 2003 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:36.578063 1789 ts_manager.cc:194] Registered new tserver with Master: 763cc216a510462c96afa6511d47448b (127.19.202.65:42439)
I20260810 22:41:36.578868 1789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:49055
Aug 10 22:41:36 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401696, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:36.830232 2149 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:36.830551 2149 heartbeater.cc:511] Master 127.19.202.126:34331 requested a full tablet report, sending...
I20260810 22:41:36.837103 1789 ts_manager.cc:194] Registered new tserver with Master: 9e3fe10a224c4772bb53831f3c00234e (127.19.202.66:41667)
I20260810 22:41:36.837904 1789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:52773
I20260810 22:41:36.839593 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:41:36.876121 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.845234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.19.202.126:34331)
0810 22:41:36.848948 (+ 3714us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:36.848952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:36.854455 (+ 5503us) server_negotiation.cc:407] Connection header received
0810 22:41:36.854600 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:36.854604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:36.854659 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:36.854758 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:36.857695 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.858551 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.859601 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.859965 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.860059 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.865801 (+ 5742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:36.865805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:36.865808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:36.865838 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:36.873806 (+ 7968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:36.874351 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:36.874354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:36.874356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:36.874467 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:36.874743 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:36.874746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:36.874748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:36.874970 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:36.875734 (+ 764us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:36.875812 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 22:41:36.875942 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3628,"thread_start_us":94,"threads_started":1}
I20260810 22:41:36.880985 1789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:41:36.886929 1789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:41:36.919764 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.903657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:42439 (local address 127.0.0.1:44780)
0810 22:41:36.906680 (+ 3023us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:36.906736 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:36.907642 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:36.908399 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:36.908405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:36.908440 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:36.908596 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.908604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.911756 (+ 3152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.911759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.917743 (+ 5984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.917755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.919154 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.919158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.919286 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.919325 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 22:41:36.919457 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 22:41:36.919597 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2853,"thread_start_us":111,"threads_started":1}
I20260810 22:41:36.920405 2329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.904504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.19.202.65:42439)
0810 22:41:36.906217 (+ 1713us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:36.906225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:36.907527 (+ 1302us) server_negotiation.cc:407] Connection header received
0810 22:41:36.907853 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:36.907859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:36.908075 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:36.908224 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:36.908749 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.910744 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.917895 (+ 7151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.918795 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.918914 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.919085 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:36.920149 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0810 22:41:36.920262 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1601,"thread_start_us":87,"threads_started":1}
I20260810 22:41:36.924074 2328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.903959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60866 (local address 127.19.202.67:38933)
0810 22:41:36.906428 (+ 2469us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:36.906439 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:36.911165 (+ 4726us) server_negotiation.cc:407] Connection header received
0810 22:41:36.911248 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:36.911253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:36.911467 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:36.911606 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:36.914032 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.915203 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.921042 (+ 5839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.922908 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.923023 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.923196 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:36.923816 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 22:41:36.923930 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2361,"thread_start_us":100,"threads_started":1}
I20260810 22:41:36.923692 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.903765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:38933 (local address 127.0.0.1:60866)
0810 22:41:36.910964 (+ 7199us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:36.910978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:36.911069 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:36.913517 (+ 2448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:36.913521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:36.913539 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:36.913885 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.913897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.915337 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.915341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.920894 (+ 5553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.920905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.923268 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.923272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.923368 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.923394 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 22:41:36.923473 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 22:41:36.923552 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7129,"thread_start_us":109,"threads_started":1}
I20260810 22:41:36.928010 2220 tablet_service.cc:1467] Processing CreateTablet for tablet 8192da59be844e01b90dd5f96402a26c (DEFAULT_TABLE table=test-table [id=0fef3362b3d549fdadccb2b1e16dacd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:36.930704 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.905671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41667 (local address 127.0.0.1:54904)
0810 22:41:36.916944 (+ 11273us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:36.916959 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:36.917053 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:36.925668 (+ 8615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:36.925672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:36.925691 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:36.925788 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.925794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.928095 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.928098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.929240 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:36.929251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.930310 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.930314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:36.930409 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.930449 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 22:41:36.930499 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 22:41:36.930566 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11202,"thread_start_us":83,"threads_started":1}
I20260810 22:41:36.930645 1930 tablet_service.cc:1467] Processing CreateTablet for tablet 8192da59be844e01b90dd5f96402a26c (DEFAULT_TABLE table=test-table [id=0fef3362b3d549fdadccb2b1e16dacd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:36.931406 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:36.905848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54904 (local address 127.19.202.66:41667)
0810 22:41:36.925100 (+ 19252us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:36.925107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:36.925136 (+ 29us) server_negotiation.cc:407] Connection header received
0810 22:41:36.925225 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:36.925232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:36.925498 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:36.925944 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:36.926099 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.927940 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.929394 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:36.930169 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:36.930984 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:36.931130 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:36.931181 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 22:41:36.931269 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19127,"thread_start_us":81,"threads_started":1}
I20260810 22:41:36.931771 1930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8192da59be844e01b90dd5f96402a26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:36.933151 2220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8192da59be844e01b90dd5f96402a26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:36.934005 2076 tablet_service.cc:1467] Processing CreateTablet for tablet 8192da59be844e01b90dd5f96402a26c (DEFAULT_TABLE table=test-table [id=0fef3362b3d549fdadccb2b1e16dacd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:36.934940 2076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8192da59be844e01b90dd5f96402a26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:36.953212 2332 tablet_bootstrap.cc:492] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Bootstrap starting.
I20260810 22:41:36.955705 2332 tablet_bootstrap.cc:654] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:36.956662 2332 log.cc:824] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:36.957538 2333 tablet_bootstrap.cc:492] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b: Bootstrap starting.
I20260810 22:41:36.959848 2333 tablet_bootstrap.cc:654] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:36.960630 2333 log.cc:824] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:36.962440 2334 tablet_bootstrap.cc:492] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Bootstrap starting.
I20260810 22:41:36.963233 2333 tablet_bootstrap.cc:492] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b: No bootstrap required, opened a new log
I20260810 22:41:36.963418 2333 ts_tablet_manager.cc:1397] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 22:41:36.964707 2334 tablet_bootstrap.cc:654] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:36.969101 2332 tablet_bootstrap.cc:492] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: No bootstrap required, opened a new log
I20260810 22:41:36.969306 2332 ts_tablet_manager.cc:1397] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.004s
I20260810 22:41:36.969694 2334 log.cc:824] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:36.976504 2333 raft_consensus.cc:348] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:36.977682 2332 raft_consensus.cc:348] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:36.978044 2332 raft_consensus.cc:374] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:36.978142 2332 raft_consensus.cc:729] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a558354a504f5eadb48bed37964f89, State: Initialized, Role: FOLLOWER
I20260810 22:41:36.978296 2333 raft_consensus.cc:374] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:36.978667 2333 raft_consensus.cc:729] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763cc216a510462c96afa6511d47448b, State: Initialized, Role: FOLLOWER
I20260810 22:41:36.979274 2332 consensus_queue.cc:260] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:36.979768 2333 consensus_queue.cc:260] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:36.981086 2332 ts_tablet_manager.cc:1428] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260810 22:41:36.982061 2333 ts_tablet_manager.cc:1428] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260810 22:41:36.982658 2003 heartbeater.cc:503] Master 127.19.202.126:34331 was elected leader, sending a full tablet report...
I20260810 22:41:36.986730 2295 heartbeater.cc:503] Master 127.19.202.126:34331 was elected leader, sending a full tablet report...
I20260810 22:41:36.994508 2334 tablet_bootstrap.cc:492] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: No bootstrap required, opened a new log
I20260810 22:41:36.994747 2334 ts_tablet_manager.cc:1397] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Time spent bootstrapping tablet: real 0.032s user 0.006s sys 0.000s
I20260810 22:41:36.998796 2334 raft_consensus.cc:348] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:36.999101 2334 raft_consensus.cc:374] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:36.999145 2334 raft_consensus.cc:729] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e3fe10a224c4772bb53831f3c00234e, State: Initialized, Role: FOLLOWER
I20260810 22:41:36.999822 2334 consensus_queue.cc:260] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:37.001058 2334 ts_tablet_manager.cc:1428] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:41:37.001497 2149 heartbeater.cc:503] Master 127.19.202.126:34331 was elected leader, sending a full tablet report...
W20260810 22:41:37.045418 2004 tablet.cc:2369] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:41:37.055682 2150 tablet.cc:2369] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:41:37.069383 2296 tablet.cc:2369] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 22:41:37 dist-test-slave-1x4g krb5kdc[693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401696, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for HTTP/127.19.202.84@KRBTEST.COM
I20260810 22:41:37.331928 2340 raft_consensus.cc:482] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:37.332165 2340 raft_consensus.cc:504] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:37.333505 2340 leader_election.cc:288] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14a558354a504f5eadb48bed37964f89 (127.19.202.67:38933), 9e3fe10a224c4772bb53831f3c00234e (127.19.202.66:41667)
I20260810 22:41:37.337937 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:37.333981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:50565 (local address 127.19.202.66:41667)
0810 22:41:37.334132 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:37.334138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:37.334315 (+ 177us) server_negotiation.cc:407] Connection header received
0810 22:41:37.334447 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:37.334452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:37.334501 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:37.334590 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:37.334894 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.335455 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.336615 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.337242 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.337374 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:37.337429 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:37.337730 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 22:41:37.337801 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:41:37.337934 2328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:37.333837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:50933 (local address 127.19.202.67:38933)
0810 22:41:37.334089 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:37.334097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:37.334141 (+ 44us) server_negotiation.cc:407] Connection header received
0810 22:41:37.334328 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:37.334336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:37.334386 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:37.334470 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:37.334789 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.335388 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.336670 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.337282 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.337390 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:37.337445 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:37.337729 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 22:41:37.337801 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260810 22:41:37.338075 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:37.333830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:41667 (local address 127.19.202.65:50565)
0810 22:41:37.334205 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:37.334222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:37.334350 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:37.334619 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:37.334623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:37.334639 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:37.334773 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:37.334778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.335569 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.335575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:37.336473 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:37.336482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.337401 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.337404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:37.337551 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:37.337614 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 22:41:37.337705 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 22:41:37.337865 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":55,"thread_start_us":115,"threads_started":1}
I20260810 22:41:37.338083 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:37.333722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:38933 (local address 127.19.202.65:50933)
0810 22:41:37.334050 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:37.334066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:37.334206 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:37.334495 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:37.334498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:37.334515 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:37.334670 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:37.334676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.335502 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.335507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:37.336518 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:37.336528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.337401 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.337404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:37.337554 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:37.337612 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 22:41:37.337699 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 22:41:37.337870 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":228,"thread_start_us":131,"threads_started":1}
I20260810 22:41:37.338794 2223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192da59be844e01b90dd5f96402a26c" candidate_uuid: "763cc216a510462c96afa6511d47448b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14a558354a504f5eadb48bed37964f89" is_pre_election: true
I20260810 22:41:37.338801 2099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192da59be844e01b90dd5f96402a26c" candidate_uuid: "763cc216a510462c96afa6511d47448b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e3fe10a224c4772bb53831f3c00234e" is_pre_election: true
I20260810 22:41:37.339143 2099 raft_consensus.cc:2454] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763cc216a510462c96afa6511d47448b in term 0.
I20260810 22:41:37.339143 2223 raft_consensus.cc:2454] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763cc216a510462c96afa6511d47448b in term 0.
I20260810 22:41:37.339680 1883 leader_election.cc:302] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 763cc216a510462c96afa6511d47448b, 9e3fe10a224c4772bb53831f3c00234e; no voters:
I20260810 22:41:37.340035 2340 raft_consensus.cc:2781] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:41:37.340139 2340 raft_consensus.cc:482] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:41:37.340204 2340 raft_consensus.cc:3037] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:37.341302 2340 raft_consensus.cc:504] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:37.341679 2340 leader_election.cc:288] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [CANDIDATE]: Term 1 election: Requested vote from peers 14a558354a504f5eadb48bed37964f89 (127.19.202.67:38933), 9e3fe10a224c4772bb53831f3c00234e (127.19.202.66:41667)
I20260810 22:41:37.342100 2223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192da59be844e01b90dd5f96402a26c" candidate_uuid: "763cc216a510462c96afa6511d47448b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14a558354a504f5eadb48bed37964f89"
I20260810 22:41:37.342190 2099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192da59be844e01b90dd5f96402a26c" candidate_uuid: "763cc216a510462c96afa6511d47448b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e3fe10a224c4772bb53831f3c00234e"
I20260810 22:41:37.342240 2223 raft_consensus.cc:3037] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:37.342296 2099 raft_consensus.cc:3037] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:37.343585 2099 raft_consensus.cc:2454] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763cc216a510462c96afa6511d47448b in term 1.
I20260810 22:41:37.343585 2223 raft_consensus.cc:2454] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763cc216a510462c96afa6511d47448b in term 1.
I20260810 22:41:37.343945 1884 leader_election.cc:302] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a558354a504f5eadb48bed37964f89, 763cc216a510462c96afa6511d47448b; no voters:
I20260810 22:41:37.344172 2340 raft_consensus.cc:2781] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:41:37.344425 2340 raft_consensus.cc:686] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [term 1 LEADER]: Becoming Leader. State: Replica: 763cc216a510462c96afa6511d47448b, State: Running, Role: LEADER
I20260810 22:41:37.344921 2340 consensus_queue.cc:237] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } }
I20260810 22:41:37.348176 1789 catalog_manager.cc:5495] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b reported cstate change: term changed from 0 to 1, leader changed from <none> to 763cc216a510462c96afa6511d47448b (127.19.202.65). New cstate: current_term: 1 leader_uuid: "763cc216a510462c96afa6511d47448b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763cc216a510462c96afa6511d47448b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 42439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260810 22:41:37.361284 2329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:37.357885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.19.202.65:42439)
0810 22:41:37.358007 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:37.358010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:37.358023 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:41:37.358159 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:37.358162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:37.358208 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:37.358292 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:41:37.358534 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.359061 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.359843 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:37.360055 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:37.360123 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:37.360385 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:41:37.360856 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:41:37.360977 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:37.361038 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 22:41:37.361085 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:41:37.366710 2223 raft_consensus.cc:1261] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 1 FOLLOWER]: Refusing update from remote peer 763cc216a510462c96afa6511d47448b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:37.366756 2099 raft_consensus.cc:1261] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 1 FOLLOWER]: Refusing update from remote peer 763cc216a510462c96afa6511d47448b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:37.367615 2354 consensus_queue.cc:1035] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e3fe10a224c4772bb53831f3c00234e" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:41:37.367785 2340 consensus_queue.cc:1035] T 8192da59be844e01b90dd5f96402a26c P 763cc216a510462c96afa6511d47448b [LEADER]: Connected to new peer: Peer: permanent_uuid: "14a558354a504f5eadb48bed37964f89" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 38933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:41:37.375806 2358 mvcc.cc:205] Tried to move back new op lower bound from 7317101352407400448 to 7317101352327393280. Current Snapshot: MvccSnapshot[applied={T|T < 7317101352407400448}]
I20260810 22:41:37.375806 2359 mvcc.cc:205] Tried to move back new op lower bound from 7317101352407400448 to 7317101352327393280. Current Snapshot: MvccSnapshot[applied={T|T < 7317101352407400448}]
I20260810 22:41:37.383885 1787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:41:37.384068 1787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:41:37.386674 1787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6071d82b64504c71adb3d476e115b283: Sending DeleteTablet for 3 replicas of tablet 8192da59be844e01b90dd5f96402a26c
I20260810 22:41:37.387459 2220 tablet_service.cc:1514] Processing DeleteTablet for tablet 8192da59be844e01b90dd5f96402a26c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:37 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:60866
I20260810 22:41:37.387617 1930 tablet_service.cc:1514] Processing DeleteTablet for tablet 8192da59be844e01b90dd5f96402a26c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:37 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:44780
I20260810 22:41:37.388051 2367 tablet_replica.cc:330] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: stopping tablet replica
I20260810 22:41:37.388084 2076 tablet_service.cc:1514] Processing DeleteTablet for tablet 8192da59be844e01b90dd5f96402a26c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:37 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:54904
I20260810 22:41:37.388163 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 1831
I20260810 22:41:37.388427 2367 raft_consensus.cc:2229] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:41:37.388723 2367 raft_consensus.cc:2258] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:41:37.390179 2367 ts_tablet_manager.cc:1905] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:41:37.393702 2367 ts_tablet_manager.cc:1918] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:41:37.393787 2367 log.cc:1196] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/8192da59be844e01b90dd5f96402a26c
I20260810 22:41:37.394112 2367 ts_tablet_manager.cc:1939] T 8192da59be844e01b90dd5f96402a26c P 14a558354a504f5eadb48bed37964f89: Deleting consensus metadata
I20260810 22:41:37.394402 2369 tablet_replica.cc:330] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: stopping tablet replica
I20260810 22:41:37.394708 2369 raft_consensus.cc:2229] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:41:37.395017 2369 raft_consensus.cc:2258] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:41:37.395311 1770 catalog_manager.cc:4841] TS 14a558354a504f5eadb48bed37964f89 (127.19.202.67:38933): tablet 8192da59be844e01b90dd5f96402a26c (table test-table [id=0fef3362b3d549fdadccb2b1e16dacd4]) successfully deleted
I20260810 22:41:37.396550 2369 ts_tablet_manager.cc:1905] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:41:37.399930 2369 ts_tablet_manager.cc:1918] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:41:37.400044 2369 log.cc:1196] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/8192da59be844e01b90dd5f96402a26c
I20260810 22:41:37.400409 2369 ts_tablet_manager.cc:1939] T 8192da59be844e01b90dd5f96402a26c P 9e3fe10a224c4772bb53831f3c00234e: Deleting consensus metadata
W20260810 22:41:37.401423 1769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:42439 (error 108)
I20260810 22:41:37.401860 1769 catalog_manager.cc:4841] TS 9e3fe10a224c4772bb53831f3c00234e (127.19.202.66:41667): tablet 8192da59be844e01b90dd5f96402a26c (table test-table [id=0fef3362b3d549fdadccb2b1e16dacd4]) successfully deleted
I20260810 22:41:37.402359 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2007
I20260810 22:41:37.402415 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:37.402132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:42439 (local address 127.0.0.1:44794)
0810 22:41:37.402276 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:37.402348 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:42439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 22:41:37.405822 1769 catalog_manager.cc:4584] TS 763cc216a510462c96afa6511d47448b (127.19.202.65:42439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8192da59be844e01b90dd5f96402a26c: Network error: Client connection negotiation failed: client connection to 127.19.202.65:42439: connect: Connection refused (error 111)
I20260810 22:41:37.410820 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2152
I20260810 22:41:37.418264 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 1750
I20260810 22:41:37.426924 20265 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 22:41:37.771092 20265 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 22:41:37.771238 20265 mini_ranger.cc:66] Stopping Ranger...
I20260810 22:41:38.115689 20265 mini_ranger.cc:68] Stopped Ranger
2026-08-10 22:41:38.115 UTC [724] LOG: received smart shutdown request
2026-08-10 22:41:38.120 UTC [724] LOG: background worker "logical replication launcher" (PID 736) exited with exit code 1
2026-08-10 22:41:38.120 UTC [728] LOG: shutting down
2026-08-10 22:41:38.291 UTC [724] LOG: database system is shut down
2026-08-10T22:41:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47670 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:41:38 dist-test-slave-1x4g krb5kdc[2380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:41:38 dist-test-slave-1x4g krb5kdc[2380](info): set up 2 sockets
Aug 10 22:41:38 dist-test-slave-1x4g krb5kdc[2380](info): commencing operation
krb5kdc: starting...
W20260810 22:41:39.503103 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:41:39 dist-test-slave-1x4g krb5kdc[2380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:41:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:41:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:41:39.623610 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:35437
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:35437,127.19.202.125:39749,127.19.202.124:43319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:39.733178 2394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:39.733441 2394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:39.733479 2394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:39.737500 2394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:39.737550 2394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:39.737571 2394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:39.737628 2394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:39.737648 2394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:39.742799 2394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:35437,127.19.202.125:39749,127.19.202.124:43319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:39.743821 2394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:39.744912 2394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:39.753486 2399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:39.753535 2402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:39.753741 2394 server_base.cc:1034] running on GCE node
W20260810 22:41:39.753546 2400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:39.754554 2394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:39.755611 2394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:39.756834 2394 hybrid_clock.cc:648] HybridClock initialized: now 1786401699756801 us; error 44 us; skew 500 ppm
Aug 10 22:41:39 dist-test-slave-1x4g krb5kdc[2380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:39.759595 2394 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:41:39.760449 2394 webserver.cc:460] Webserver started at http://127.19.202.126:46727/ using document root <none> and password file <none>
I20260810 22:41:39.761032 2394 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:39.761096 2394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:39.761261 2394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:39.763186 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "d04f59fad0ec40f987a35dece162f585"
format_stamp: "Formatted at 2026-08-10 22:41:39 on dist-test-slave-1x4g"
server_key: "99a5891cf686d595a3c50ca6579de4ae"
server_key_iv: "77bbc2bfc7c893855cd1fa7d7b87e9a4"
server_key_version: "encryptionkey@0"
I20260810 22:41:39.763731 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "d04f59fad0ec40f987a35dece162f585"
format_stamp: "Formatted at 2026-08-10 22:41:39 on dist-test-slave-1x4g"
server_key: "99a5891cf686d595a3c50ca6579de4ae"
server_key_iv: "77bbc2bfc7c893855cd1fa7d7b87e9a4"
server_key_version: "encryptionkey@0"
I20260810 22:41:39.767278 2394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 22:41:39.769649 2410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:39.770848 2394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:41:39.770977 2394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "d04f59fad0ec40f987a35dece162f585"
format_stamp: "Formatted at 2026-08-10 22:41:39 on dist-test-slave-1x4g"
server_key: "99a5891cf686d595a3c50ca6579de4ae"
server_key_iv: "77bbc2bfc7c893855cd1fa7d7b87e9a4"
server_key_version: "encryptionkey@0"
I20260810 22:41:39.771082 2394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:39.791193 2394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:39.794086 2394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:39.794305 2394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:39.801828 2394 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:35437
I20260810 22:41:39.802127 2467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:35437 every 8 connection(s)
I20260810 22:41:39.802836 2394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:41:39.805614 2468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:39.809671 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2394
I20260810 22:41:39.809322 2468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:41:39.809825 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:41:39.810142 20265 external_mini_cluster.cc:1428] Setting key b38fa336dcacffbf89ef268c7db7ce84
WARNING: no policy specified for kudu/127.19.202.125@KRBTEST.COM; defaulting to no policy
Aug 10 22:41:39 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.125@KRBTEST.COM" created.
I20260810 22:41:39.824996 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.811848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:35437 (local address 127.0.0.1:54340)
0810 22:41:39.812401 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:39.812447 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:39.813334 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:39.814015 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:39.814021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:39.814460 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:39.814692 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:39.814704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:39.815891 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:39.815894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:39.816632 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:39.816641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:39.817319 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:39.817322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:39.817416 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:39.818010 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:39.818026 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:39.819651 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:39.821715 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:39.821723 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:39.821731 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:39.822027 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:39.822473 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:39.822477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:39.822479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:39.822651 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:39.823247 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:39.823257 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:39.823466 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 22:41:39.823669 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 22:41:39.824200 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":185,"thread_start_us":134,"threads_started":1}
I20260810 22:41:39.824999 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.812817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.19.202.126:35437)
0810 22:41:39.813098 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:39.813109 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:39.813156 (+ 47us) server_negotiation.cc:407] Connection header received
0810 22:41:39.813646 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:39.813651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:39.813822 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:39.814016 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:39.814840 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:39.815749 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:39.816753 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:39.817166 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:39.817339 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:39.819808 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:39.819813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:39.819823 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:39.819843 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:39.821615 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:39.822169 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:39.822178 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:39.822183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:39.822328 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:39.822767 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:39.822769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:39.822770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:39.823116 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:39.823222 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:39.823738 (+ 516us) server_negotiation.cc:299] Negotiation successful
0810 22:41:39.824121 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":188,"spinlock_wait_cycles":7040,"thread_start_us":102,"threads_started":1}
I20260810 22:41:39.827718 2468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:41:39.828467 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.828217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37688)
0810 22:41:39.828332 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:39.828413 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 22:41:39.828696 2414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
I20260810 22:41:39.829557 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.829358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37698)
0810 22:41:39.829460 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:39.829510 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 22:41:39.829728 2468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:39749: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab.
Entry for principal kudu/127.19.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab.
I20260810 22:41:39.839504 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:39749
--webserver_interface=127.19.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab
--principal=kudu/127.19.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:35437,127.19.202.125:39749,127.19.202.124:43319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:39.878965 2468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } attempt: 1
I20260810 22:41:39.879978 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.879539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37714)
0810 22:41:39.879744 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:39.879837 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 22:41:39.880865 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.880548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37730)
0810 22:41:39.880682 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:39.880749 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 22:41:39.881080 2468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:39749: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
I20260810 22:41:39.949257 2468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } attempt: 2
W20260810 22:41:39.949438 2475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:39.949724 2475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:39.949827 2475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 22:41:39.950157 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.949826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37736)
0810 22:41:39.949990 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:39.950073 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 22:41:39.950860 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:39.950647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37744)
0810 22:41:39.950753 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:39.950801 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:41:39.951004 2468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:39749: Network error: Client connection negotiation failed: client connection to 127.19.202.125:39749: connect: Connection refused (error 111)
W20260810 22:41:39.954241 2475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:39.954319 2475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:39.954365 2475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:39.954396 2475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:39.954433 2475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:39.959589 2475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:35437,127.19.202.125:39749,127.19.202.124:43319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab
--principal=kudu/127.19.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:39.960590 2475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:39.961630 2475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:39.970162 2480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:39.970269 2475 server_base.cc:1034] running on GCE node
W20260810 22:41:39.970309 2481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:39.970181 2483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:39.971232 2475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:39.972271 2475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:39.973483 2475 hybrid_clock.cc:648] HybridClock initialized: now 1786401699973478 us; error 29 us; skew 500 ppm
Aug 10 22:41:39 dist-test-slave-1x4g krb5kdc[2380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:39.976744 2475 init.cc:377] Logged in from keytab as kudu/127.19.202.125@KRBTEST.COM (short username kudu)
I20260810 22:41:39.977946 2475 webserver.cc:460] Webserver started at http://127.19.202.125:35375/ using document root <none> and password file <none>
I20260810 22:41:39.978664 2475 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:39.978766 2475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:39.978992 2475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:39.981105 2475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data/instance:
uuid: "f67ed21fdb32491687c62572eaf64896"
format_stamp: "Formatted at 2026-08-10 22:41:39 on dist-test-slave-1x4g"
server_key: "75ed8aa13f1d5233daec2ba64b618723"
server_key_iv: "beca18d21bfa59316175594941932084"
server_key_version: "encryptionkey@0"
I20260810 22:41:39.981639 2475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/wal/instance:
uuid: "f67ed21fdb32491687c62572eaf64896"
format_stamp: "Formatted at 2026-08-10 22:41:39 on dist-test-slave-1x4g"
server_key: "75ed8aa13f1d5233daec2ba64b618723"
server_key_iv: "beca18d21bfa59316175594941932084"
server_key_version: "encryptionkey@0"
I20260810 22:41:39.985314 2475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:41:39.987592 2491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:39.988754 2475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:41:39.988930 2475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/wal
uuid: "f67ed21fdb32491687c62572eaf64896"
format_stamp: "Formatted at 2026-08-10 22:41:39 on dist-test-slave-1x4g"
server_key: "75ed8aa13f1d5233daec2ba64b618723"
server_key_iv: "beca18d21bfa59316175594941932084"
server_key_version: "encryptionkey@0"
I20260810 22:41:39.989051 2475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:40.009649 2475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:40.012403 2475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:40.012629 2475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:40.019753 2475 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.125:39749
I20260810 22:41:40.020064 2552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.125:39749 every 8 connection(s)
I20260810 22:41:40.020813 2475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
I20260810 22:41:40.023599 2553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:40.025744 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2475
I20260810 22:41:40.025889 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-1/wal/instance
I20260810 22:41:40.026175 20265 external_mini_cluster.cc:1428] Setting key 5fc7a08b15377819f0c6018c614bad09
I20260810 22:41:40.028268 2553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.202.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.124@KRBTEST.COM" created.
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:40.042177 2468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } attempt: 3
I20260810 22:41:40.043509 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.030231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.19.202.126:35437)
0810 22:41:40.030389 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.030394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.031219 (+ 825us) server_negotiation.cc:407] Connection header received
0810 22:41:40.032112 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.032118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.032187 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.032301 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.033507 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.034368 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.035588 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.035829 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.035935 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.038914 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.038922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.038924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.038950 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.040520 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.041121 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.041124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.041126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.041209 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.041568 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.041572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.041573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.041756 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.041868 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.043181 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.043323 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 22:41:40.045037 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.030596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:35437 (local address 127.0.0.1:54354)
0810 22:41:40.031048 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.031100 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.031898 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.032441 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.032446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.032747 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.033309 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.033322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.034508 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.034511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.035473 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.035483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.036047 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.036054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.036265 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.036888 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.036904 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.038741 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.040649 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.040655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.040670 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.041015 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.041310 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.041313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.041315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.041463 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.041869 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.041876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.042156 (+ 280us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.043721 (+ 1565us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.044072 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":121,"threads_started":1}
I20260810 22:41:40.046406 2553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
I20260810 22:41:40.053980 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.042638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37746)
0810 22:41:40.043322 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.043340 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.043445 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.043936 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.043943 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.044130 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.044249 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.044257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.045583 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.045586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.046487 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.046503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.047352 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.047357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.047489 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.047956 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.047972 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.049862 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.052095 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.052098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.052099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.052311 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.052661 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.052663 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.052664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.052757 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.053474 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.053476 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.053561 (+ 85us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.053693 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.053850 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581}
I20260810 22:41:40.054255 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.042882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.19.202.125:39749)
0810 22:41:40.043285 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.043297 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.043427 (+ 130us) server_negotiation.cc:407] Connection header received
0810 22:41:40.043551 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.043557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.043765 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.043902 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.044379 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.045421 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.046639 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.047240 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.047342 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.049992 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.050002 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.050015 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.050047 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.051966 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.052443 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.052450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.052453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.052563 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.052932 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.052937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.052940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.053290 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.053454 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.053833 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.054046 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab.
Entry for principal kudu/127.19.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab.
I20260810 22:41:40.055543 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:43319
--webserver_interface=127.19.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab
--principal=kudu/127.19.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:46689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:35437,127.19.202.125:39749,127.19.202.124:43319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:40.056356 2468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:41:40.056562 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.046934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37748)
0810 22:41:40.047033 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.047047 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.047117 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.047562 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.047564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.047685 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.047765 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.047770 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.048680 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.048684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.049270 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.049279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.049793 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.049797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.049892 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.050344 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.050364 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.051910 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.054810 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.054816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.054821 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.055097 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.055366 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.055368 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.055370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.055456 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.056023 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.056026 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.056106 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.056229 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.056383 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":3}
I20260810 22:41:40.057032 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.047082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37748 (local address 127.19.202.125:39749)
0810 22:41:40.047359 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.047368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.047378 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:41:40.047407 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.047409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.047450 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.047535 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.047967 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.048571 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.049381 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.049679 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.049783 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.052055 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.052061 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.052065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.052091 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.054687 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.055201 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.055204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.055205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.055280 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.055546 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.055550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.055551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.055908 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.056030 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.056302 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.056529 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":220,"thread_start_us":117,"threads_started":1}
I20260810 22:41:40.057196 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.056854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48494)
0810 22:41:40.057047 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.057108 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260810 22:41:40.057672 2553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:41:40.058095 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.057711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48502)
0810 22:41:40.057960 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.058020 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260810 22:41:40.058318 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.058081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48508)
0810 22:41:40.058194 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.058259 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 22:41:40.058346 2468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43319: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
W20260810 22:41:40.058496 2494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
I20260810 22:41:40.059269 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.059077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48512)
0810 22:41:40.059174 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.059214 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 22:41:40.059420 2553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43319: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
I20260810 22:41:40.089507 2468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } attempt: 1
I20260810 22:41:40.090358 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.090024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48524)
0810 22:41:40.090180 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.090250 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 22:41:40.091110 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.090867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48538)
0810 22:41:40.090998 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.091048 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 22:41:40.091256 2468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43319: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
I20260810 22:41:40.108739 2553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } attempt: 1
I20260810 22:41:40.109586 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.109311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48552)
0810 22:41:40.109464 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.109520 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 22:41:40.110302 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.110088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48554)
0810 22:41:40.110196 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.110249 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 22:41:40.110455 2553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43319: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
I20260810 22:41:40.166476 2468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } attempt: 2
I20260810 22:41:40.167356 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.167067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48570)
0810 22:41:40.167215 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.167278 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 22:41:40.168061 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.167851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48574)
0810 22:41:40.167957 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.168012 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 22:41:40.168186 2468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43319: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
W20260810 22:41:40.168762 2560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:40.169085 2560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:40.169178 2560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:40.173071 2560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:40.173142 2560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:40.173184 2560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:40.173224 2560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:40.173254 2560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:40.178610 2553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } attempt: 2
I20260810 22:41:40.178961 2560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:46689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:35437,127.19.202.125:39749,127.19.202.124:43319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab
--principal=kudu/127.19.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:43319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:40.179441 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.179076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48576)
0810 22:41:40.179304 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.179366 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260810 22:41:40.180181 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.179957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48580)
0810 22:41:40.180065 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.180115 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 22:41:40.180287 2560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260810 22:41:40.180315 2553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43319: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43319: connect: Connection refused (error 111)
I20260810 22:41:40.181325 2560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:40.189337 2567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:40.189364 2570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:40.189337 2568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:40.189685 2560 server_base.cc:1034] running on GCE node
I20260810 22:41:40.190224 2560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:40.191300 2560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:40.192512 2560 hybrid_clock.cc:648] HybridClock initialized: now 1786401700192500 us; error 28 us; skew 500 ppm
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401700, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:40.195441 2560 init.cc:377] Logged in from keytab as kudu/127.19.202.124@KRBTEST.COM (short username kudu)
I20260810 22:41:40.196265 2560 webserver.cc:460] Webserver started at http://127.19.202.124:43583/ using document root <none> and password file <none>
I20260810 22:41:40.196921 2560 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:40.197023 2560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:40.197268 2560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:40.199333 2560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data/instance:
uuid: "b2984815e6ee47dfaa54a38f67ebb88f"
format_stamp: "Formatted at 2026-08-10 22:41:40 on dist-test-slave-1x4g"
server_key: "9a4055b7c1824048397f129419581e36"
server_key_iv: "0645c84e7b218216c65ea211e4e21cb1"
server_key_version: "encryptionkey@0"
I20260810 22:41:40.199967 2560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/wal/instance:
uuid: "b2984815e6ee47dfaa54a38f67ebb88f"
format_stamp: "Formatted at 2026-08-10 22:41:40 on dist-test-slave-1x4g"
server_key: "9a4055b7c1824048397f129419581e36"
server_key_iv: "0645c84e7b218216c65ea211e4e21cb1"
server_key_version: "encryptionkey@0"
I20260810 22:41:40.203809 2560 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 22:41:40.206466 2578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:40.207811 2560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 22:41:40.207970 2560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/wal
uuid: "b2984815e6ee47dfaa54a38f67ebb88f"
format_stamp: "Formatted at 2026-08-10 22:41:40 on dist-test-slave-1x4g"
server_key: "9a4055b7c1824048397f129419581e36"
server_key_iv: "0645c84e7b218216c65ea211e4e21cb1"
server_key_version: "encryptionkey@0"
I20260810 22:41:40.208093 2560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:40.228049 2560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:40.231411 2560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:40.231649 2560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:40.238991 2560 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.124:43319
I20260810 22:41:40.239331 2637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.124:43319 every 8 connection(s)
I20260810 22:41:40.240073 2560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
I20260810 22:41:40.242723 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2560
I20260810 22:41:40.242875 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786401536977653-20265-0/minicluster-data/master-2/wal/instance
I20260810 22:41:40.242939 2638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:40.243173 20265 external_mini_cluster.cc:1428] Setting key b06a7f9deba86a62135538be3372341c
I20260810 22:41:40.247498 2638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401700, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:40.258224 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.244598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.19.202.124:43319)
0810 22:41:40.245068 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.245082 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.245116 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:41:40.245708 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.245723 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.246074 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.246457 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.247268 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.248333 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.249305 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.249929 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.250116 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.252938 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.252945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.252957 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.252988 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.254972 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.255527 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.255532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.255538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.255676 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.255967 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.255974 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.255975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.256347 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.256600 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.256905 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.257188 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":160,"threads_started":1}
I20260810 22:41:40.259080 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.249231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.19.202.126:35437)
0810 22:41:40.249373 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.249378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.249642 (+ 264us) server_negotiation.cc:407] Connection header received
0810 22:41:40.249811 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.249819 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.249862 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.249948 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.250722 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.251517 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.252431 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.252838 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.252935 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.255564 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.255568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.255572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.255603 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.257291 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.257779 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.257782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.257783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.257866 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.258129 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.258131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.258132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.258316 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.258428 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.258827 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.258938 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 22:41:40.259102 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.249140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:35437 (local address 127.0.0.1:54362)
0810 22:41:40.249523 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.249561 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.249701 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.249998 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.250004 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.250346 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.250574 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.250584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.251645 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.251648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.252260 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.252269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.252951 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.252954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.253088 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.253688 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.253706 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.255424 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.257397 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.257406 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.257411 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.257677 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.257950 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.257953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.257955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.258044 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.258486 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.258491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.258611 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.258753 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.258934 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":101,"threads_started":1}
I20260810 22:41:40.260143 2638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401700, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:40.267342 2468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } attempt: 3
I20260810 22:41:40.269999 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.260647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:39749 (local address 127.0.0.1:37754)
0810 22:41:40.260829 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.260840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.260909 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.261152 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.261154 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.261325 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.261441 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.261448 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.262358 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.262361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.263095 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.263105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.263590 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.263593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.263656 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.264120 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.264138 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.265920 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.268125 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.268132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.268133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.268395 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.268721 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.268724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.268726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.268846 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.269500 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.269502 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.269573 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.269679 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.269818 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 22:41:40.270033 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.260741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37754 (local address 127.19.202.125:39749)
0810 22:41:40.260909 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.260924 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.260939 (+ 15us) server_negotiation.cc:407] Connection header received
0810 22:41:40.260983 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.260986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.261033 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.261112 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.261565 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.262250 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.263206 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.263479 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.263578 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.266079 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.266082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.266084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.266106 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.268017 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.268501 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.268504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.268505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.268614 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.268955 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.268959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.268960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.269361 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.269454 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.269750 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.269895 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 22:41:40.270655 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.260640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.19.202.126:35437)
0810 22:41:40.260997 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.261001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.261011 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:41:40.261042 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.261044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.261085 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.261155 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.261870 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.262500 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.263485 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.263767 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.263879 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.266134 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.266138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.266140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.266164 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.268085 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.268985 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.268988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.268989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.269074 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.269388 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.269393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.269395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.269601 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.269811 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.270410 (+ 599us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.270513 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":292}
I20260810 22:41:40.271328 2638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:41:40.271458 2553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } attempt: 3
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
I20260810 22:41:40.276562 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.267760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48600)
0810 22:41:40.267871 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.267883 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.267944 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.268191 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.268196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.268365 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.268471 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.268479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.269426 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.269432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.270249 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.270261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.270842 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.270846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.270927 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.271410 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.271424 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.272838 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.274991 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.275006 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.275008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.275252 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.275538 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.275542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.275544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.275631 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.276058 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.276061 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.276152 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.276289 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.276407 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":3}
I20260810 22:41:40.277201 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.267847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.19.202.124:43319)
0810 22:41:40.267960 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.267965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.267998 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:41:40.268035 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.268041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.268081 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.268175 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.268591 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.269331 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.270384 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.270745 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.270850 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.272981 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.272984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.272987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.273008 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.274854 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.275351 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.275354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.275356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.275437 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.275731 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.275735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.275736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.275918 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.276010 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.276908 (+ 898us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.277054 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Aug 10 22:41:40 dist-test-slave-1x4g krb5kdc[2380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401700, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
I20260810 22:41:40.281169 2468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: Bootstrap starting.
I20260810 22:41:40.284168 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.272337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.19.202.125:39749)
0810 22:41:40.272428 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.272432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.272586 (+ 154us) server_negotiation.cc:407] Connection header received
0810 22:41:40.272775 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.272780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.272847 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.272898 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.273881 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.274702 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.275777 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.276321 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.276389 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.280497 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.280500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.280502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.280522 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.281938 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.282390 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.282397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.282400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.282482 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.283096 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.283100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.283102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.283308 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.283482 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.283856 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.283973 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":37}
I20260810 22:41:40.284235 2468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:40.285231 2468 log.cc:824] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:40.285715 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.272581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.19.202.124:43319)
0810 22:41:40.273060 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.273069 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.273079 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:41:40.273109 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.273112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.273147 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.273234 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.274264 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.275011 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.275931 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.276262 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.276352 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.279973 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.279976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.279978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.279996 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.283411 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.283960 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.283963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.283965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.284060 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.284587 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.284590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.284591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.284811 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.284897 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.285434 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.285541 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":405,"spinlock_wait_cycles":1408,"thread_start_us":95,"threads_started":1}
I20260810 22:41:40.285881 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.271816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48606)
0810 22:41:40.272066 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.272079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.272142 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.272627 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.272630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.272821 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.272958 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.272967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.275249 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.275252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.276113 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.276122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.276754 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.276758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.276880 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.277386 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.277405 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.279899 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.283957 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.283960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.283962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.284237 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.284567 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.284573 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.284576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.284663 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.285395 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.285398 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.285499 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.285583 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.285699 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":34}
I20260810 22:41:40.286365 2643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.272043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.19.202.124:43319)
0810 22:41:40.272370 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.272379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.272392 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:41:40.272424 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.272427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.272468 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.273236 (+ 768us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.274015 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.275121 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.276229 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.276588 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.277122 (+ 534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.280513 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.280516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.280519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.280543 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.283833 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.284356 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.284359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.284360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.284447 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.284775 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.284777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.284779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.285283 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.286040 (+ 757us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.286116 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.286217 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":265,"spinlock_wait_cycles":4224,"thread_start_us":115,"threads_started":1}
I20260810 22:41:40.286525 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.272365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43319 (local address 127.0.0.1:48618)
0810 22:41:40.272866 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:40.272881 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:40.272937 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:40.273573 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:40.273578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:40.273797 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:40.273896 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.273903 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.275115 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.275118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.275834 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:40.275843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.276467 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.276471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:40.276543 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.278078 (+ 1535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:40.278094 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:40.279835 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:40.283535 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.283538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.283541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.283856 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.284155 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:40.284160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:40.284164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:40.284482 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:40.284946 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:40.284948 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:40.285022 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 22:41:40.285122 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 22:41:40.285246 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440}
I20260810 22:41:40.288378 2468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: No bootstrap required, opened a new log
I20260810 22:41:40.289155 2638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: Bootstrap starting.
I20260810 22:41:40.289621 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896: Bootstrap starting.
I20260810 22:41:40.291062 2638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:40.291677 2638 log.cc:824] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:40.291958 2553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:40.292482 2468 raft_consensus.cc:348] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.292726 2468 raft_consensus.cc:374] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:40.292793 2468 raft_consensus.cc:729] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04f59fad0ec40f987a35dece162f585, State: Initialized, Role: FOLLOWER
I20260810 22:41:40.292851 2553 log.cc:824] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:40.293322 2468 consensus_queue.cc:260] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.293474 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:41:40.293725 2638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: No bootstrap required, opened a new log
I20260810 22:41:40.294665 2468 sys_catalog.cc:564] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:40.294797 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896: No bootstrap required, opened a new log
W20260810 22:41:40.295850 2418 tablet.cc:2369] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:41:40.298900 2553 raft_consensus.cc:348] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.299121 2553 raft_consensus.cc:374] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:40.298993 2638 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.299175 2553 raft_consensus.cc:729] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f67ed21fdb32491687c62572eaf64896, State: Initialized, Role: FOLLOWER
I20260810 22:41:40.299242 2638 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:40.299306 2638 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2984815e6ee47dfaa54a38f67ebb88f, State: Initialized, Role: FOLLOWER
I20260810 22:41:40.299633 2553 consensus_queue.cc:260] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.299762 2638 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.300873 2553 sys_catalog.cc:564] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:40.300988 2638 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [sys.catalog]: configured and running, proceeding with master startup.
W20260810 22:41:40.306660 2692 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:41:40.306733 2692 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:41:40.303337 2668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.307010 2668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:41:40.308313 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.296440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.19.202.125:39749)
0810 22:41:40.296581 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.296585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.298287 (+ 1702us) server_negotiation.cc:407] Connection header received
0810 22:41:40.298460 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.298464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.298504 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.298570 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.299842 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.300574 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.302246 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.302573 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.302647 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.304515 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.304519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.304534 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.304558 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.306230 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.306669 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.306675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.306679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.306759 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.307229 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.307235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.307238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.307523 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.307633 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.308035 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.308148 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 22:41:40.312122 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.297353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.19.202.124:43319)
0810 22:41:40.297638 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.297642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.302075 (+ 4433us) server_negotiation.cc:407] Connection header received
0810 22:41:40.302205 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.302210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.302257 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.302322 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.304177 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.305075 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.306291 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.306646 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.306720 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.308323 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.308329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.308333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.308360 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.310212 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.310695 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.310700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.310705 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.310797 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.311132 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.311138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.311142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.311356 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.311490 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.311862 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.311993 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260810 22:41:40.312126 2475 master_runner.cc:263] No leader master found from master f67ed21fdb32491687c62572eaf64896
I20260810 22:41:40.312186 2475 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 22:41:40.312937 2648 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.313069 2648 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:41:40.313694 2668 raft_consensus.cc:482] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:40.313838 2668 raft_consensus.cc:504] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.314433 2560 master_runner.cc:263] No leader master found from master b2984815e6ee47dfaa54a38f67ebb88f
I20260810 22:41:40.314503 2560 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 22:41:40.314808 2668 leader_election.cc:288] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d04f59fad0ec40f987a35dece162f585 (127.19.202.126:35437), b2984815e6ee47dfaa54a38f67ebb88f (127.19.202.124:43319)
I20260810 22:41:40.299557 2394 master_runner.cc:186] Error getting master registration for 127.19.202.125:39749: OK, instance_id { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" instance_seqno: 1786401700017737 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 22:41:40.315236 2394 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 22:41:40.315331 2440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f67ed21fdb32491687c62572eaf64896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d04f59fad0ec40f987a35dece162f585" is_pre_election: true
I20260810 22:41:40.315575 2440 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f67ed21fdb32491687c62572eaf64896 in term 0.
I20260810 22:41:40.316253 2493 leader_election.cc:302] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d04f59fad0ec40f987a35dece162f585, f67ed21fdb32491687c62572eaf64896; no voters:
I20260810 22:41:40.316287 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f67ed21fdb32491687c62572eaf64896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" is_pre_election: true
I20260810 22:41:40.316540 2607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f67ed21fdb32491687c62572eaf64896 in term 0.
I20260810 22:41:40.316555 2668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:41:40.316653 2668 raft_consensus.cc:482] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:41:40.316709 2668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 0 FOLLOWER]: Advancing to term 1
W20260810 22:41:40.316733 2693 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:41:40.316831 2693 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:41:40.318192 2668 raft_consensus.cc:504] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.318868 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f67ed21fdb32491687c62572eaf64896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2984815e6ee47dfaa54a38f67ebb88f"
I20260810 22:41:40.318863 2440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f67ed21fdb32491687c62572eaf64896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d04f59fad0ec40f987a35dece162f585"
I20260810 22:41:40.318987 2440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:40.318992 2607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:40.320214 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:40.297638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.19.202.126:35437)
0810 22:41:40.298247 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:40.298252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:40.310117 (+ 11865us) server_negotiation.cc:407] Connection header received
0810 22:41:40.310327 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:40.310332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:40.310377 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:40.310448 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:40.311218 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.311916 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.313168 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:40.313514 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:40.313604 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:40.315769 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:40.315772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:40.315777 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:40.315806 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:40.318211 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.318717 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.318721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.318722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.318815 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:40.319173 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:40.319176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:40.319178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:40.319429 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:40.319545 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:40.319916 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 22:41:40.320062 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546}
I20260810 22:41:40.320300 2440 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f67ed21fdb32491687c62572eaf64896 in term 1.
I20260810 22:41:40.320329 2607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f67ed21fdb32491687c62572eaf64896 in term 1.
I20260810 22:41:40.320632 2494 leader_election.cc:302] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2984815e6ee47dfaa54a38f67ebb88f, f67ed21fdb32491687c62572eaf64896; no voters:
I20260810 22:41:40.320926 2668 leader_election.cc:288] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [CANDIDATE]: Term 1 election: Requested vote from peers d04f59fad0ec40f987a35dece162f585 (127.19.202.126:35437), b2984815e6ee47dfaa54a38f67ebb88f (127.19.202.124:43319)
I20260810 22:41:40.321058 2696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:41:40.321206 2696 raft_consensus.cc:686] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [term 1 LEADER]: Becoming Leader. State: Replica: f67ed21fdb32491687c62572eaf64896, State: Running, Role: LEADER
I20260810 22:41:40.321554 2696 consensus_queue.cc:237] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } }
I20260810 22:41:40.323388 2696 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f67ed21fdb32491687c62572eaf64896. Latest consensus state: current_term: 1 leader_uuid: "f67ed21fdb32491687c62572eaf64896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.323554 2696 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: This master's current role is: LEADER
W20260810 22:41:40.324060 2664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:41:40.324153 2664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:41:40.325066 2699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:41:40.326893 2669 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.327028 2669 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:41:40.327920 2699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:41:40.338089 2607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [term 1 FOLLOWER]: Refusing update from remote peer f67ed21fdb32491687c62572eaf64896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:40.338089 2440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [term 1 FOLLOWER]: Refusing update from remote peer f67ed21fdb32491687c62572eaf64896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:40.338814 2696 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:41:40.339699 2668 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:41:40.343605 2648 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f67ed21fdb32491687c62572eaf64896. Latest consensus state: current_term: 1 leader_uuid: "f67ed21fdb32491687c62572eaf64896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.343724 2648 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:41:40.343994 2669 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f67ed21fdb32491687c62572eaf64896. Latest consensus state: current_term: 1 leader_uuid: "f67ed21fdb32491687c62572eaf64896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.344139 2669 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:41:40.357898 2668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f67ed21fdb32491687c62572eaf64896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.358016 2668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:40.359009 2669 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f67ed21fdb32491687c62572eaf64896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.359112 2669 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:41:40.359409 2701 mvcc.cc:205] Tried to move back new op lower bound from 7317101364576800768 to 7317101364520521728. Current Snapshot: MvccSnapshot[applied={T|T < 7317101364576800768}]
I20260810 22:41:40.360283 2696 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f67ed21fdb32491687c62572eaf64896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.360407 2696 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:40.361531 2648 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f67ed21fdb32491687c62572eaf64896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04f59fad0ec40f987a35dece162f585" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35437 } } peers { permanent_uuid: "f67ed21fdb32491687c62572eaf64896" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 39749 } } peers { permanent_uuid: "b2984815e6ee47dfaa54a38f67ebb88f" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43319 } } }
I20260810 22:41:40.361649 2648 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:41:40.362212 2702 mvcc.cc:205] Tried to move back new op lower bound from 7317101364576800768 to 7317101364520521728. Current Snapshot: MvccSnapshot[applied={T|T < 7317101364576800768}]
I20260810 22:41:40.366505 2699 catalog_manager.cc:1347] Generated new cluster ID: 1efb10235472474c88b5a91ac7f3ea52
I20260810 22:41:40.366573 2699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:41:40.397835 2699 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:41:40.401090 2699 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:41:40.420315 2699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f67ed21fdb32491687c62572eaf64896: Generated new TSK 0
I20260810 22:41:40.421294 2699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:41:41.318187 2693 catalog_manager.cc:1259] Loaded cluster ID: 1efb10235472474c88b5a91ac7f3ea52
I20260810 22:41:41.318276 2693 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: loading cluster ID for follower catalog manager: success
I20260810 22:41:41.320046 2693 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: acquiring CA information for follower catalog manager: success
I20260810 22:41:41.321341 2693 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2984815e6ee47dfaa54a38f67ebb88f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 22:41:41.325349 2664 catalog_manager.cc:1259] Loaded cluster ID: 1efb10235472474c88b5a91ac7f3ea52
I20260810 22:41:41.325423 2664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: loading cluster ID for follower catalog manager: success
I20260810 22:41:41.326912 2664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: acquiring CA information for follower catalog manager: success
I20260810 22:41:41.327926 2664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d04f59fad0ec40f987a35dece162f585: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 22:41:41.490984 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2394
I20260810 22:41:41.500672 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2475
I20260810 22:41:41.509104 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2560
2026-08-10T22:41:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3096 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:41:41 dist-test-slave-1x4g krb5kdc[2716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:41:41 dist-test-slave-1x4g krb5kdc[2716](info): set up 2 sockets
Aug 10 22:41:41 dist-test-slave-1x4g krb5kdc[2716](info): commencing operation
krb5kdc: starting...
W20260810 22:41:42.608175 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:41:42 dist-test-slave-1x4g krb5kdc[2716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:41:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:41:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:41:42.734127 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41111
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:41111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:42.848618 2730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:42.848951 2730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:42.849021 2730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:42.852942 2730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:42.852993 2730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:42.853011 2730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:42.853067 2730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:42.853096 2730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:42.858423 2730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:41111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:42.859434 2730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:42.860456 2730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:42.869026 2737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:42.869148 2730 server_base.cc:1034] running on GCE node
W20260810 22:41:42.869105 2739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:42.869027 2736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:42.869881 2730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:42.870946 2730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:42.872114 2730 hybrid_clock.cc:648] HybridClock initialized: now 1786401702872097 us; error 31 us; skew 500 ppm
Aug 10 22:41:42 dist-test-slave-1x4g krb5kdc[2716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401702, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:42.880036 2730 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:41:42.881253 2730 webserver.cc:460] Webserver started at http://127.19.202.126:33965/ using document root <none> and password file <none>
I20260810 22:41:42.881824 2730 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:42.881889 2730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:42.882134 2730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:42.884183 2730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "edaaddde79524f9389c8e7ef245ac098"
format_stamp: "Formatted at 2026-08-10 22:41:42 on dist-test-slave-1x4g"
server_key: "3da334b611979f640ba5689961712f3b"
server_key_iv: "048d0ee346e55cd62395207f5c8b2352"
server_key_version: "encryptionkey@0"
I20260810 22:41:42.884827 2730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "edaaddde79524f9389c8e7ef245ac098"
format_stamp: "Formatted at 2026-08-10 22:41:42 on dist-test-slave-1x4g"
server_key: "3da334b611979f640ba5689961712f3b"
server_key_iv: "048d0ee346e55cd62395207f5c8b2352"
server_key_version: "encryptionkey@0"
I20260810 22:41:42.888418 2730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:41:42.890863 2747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:42.892089 2730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:41:42.892232 2730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "edaaddde79524f9389c8e7ef245ac098"
format_stamp: "Formatted at 2026-08-10 22:41:42 on dist-test-slave-1x4g"
server_key: "3da334b611979f640ba5689961712f3b"
server_key_iv: "048d0ee346e55cd62395207f5c8b2352"
server_key_version: "encryptionkey@0"
I20260810 22:41:42.892346 2730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:42.916268 2730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:42.919169 2730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:42.919395 2730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:42.926434 2730 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:41111
I20260810 22:41:42.926463 2802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:41111 every 8 connection(s)
I20260810 22:41:42.927361 2730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:41:42.930197 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:42.930243 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2730
I20260810 22:41:42.930358 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:41:42.930663 20265 external_mini_cluster.cc:1428] Setting key 17891e9c3bbdb54e218f42b34b5b0511
I20260810 22:41:42.936496 2803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098: Bootstrap starting.
I20260810 22:41:42.938889 2803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098: Neither blocks nor log segments found. Creating new log.
Aug 10 22:41:42 dist-test-slave-1x4g krb5kdc[2716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:42.939813 2803 log.cc:824] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:42.942127 2803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098: No bootstrap required, opened a new log
I20260810 22:41:42.945562 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:42.932046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32886 (local address 127.19.202.126:41111)
0810 22:41:42.932489 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:42.932503 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:42.932538 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:41:42.933485 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:42.933505 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:42.933838 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:42.934189 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:42.935035 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:42.935971 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:42.936962 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:42.937436 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:42.937618 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:42.940355 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:42.940360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:42.940372 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:42.940395 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:42.942453 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:42.942892 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:42.942898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:42.942904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:42.943045 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:42.943305 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:42.943308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:42.943309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:42.943792 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:42.943983 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:42.944220 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 22:41:42.944579 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":134,"threads_started":1}
I20260810 22:41:42.946787 2803 raft_consensus.cc:348] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaaddde79524f9389c8e7ef245ac098" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41111 } }
I20260810 22:41:42.946996 2803 raft_consensus.cc:374] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:42.947057 2803 raft_consensus.cc:729] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edaaddde79524f9389c8e7ef245ac098, State: Initialized, Role: FOLLOWER
I20260810 22:41:42.947525 2803 consensus_queue.cc:260] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaaddde79524f9389c8e7ef245ac098" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41111 } }
I20260810 22:41:42.947713 2803 raft_consensus.cc:388] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:41:42.947816 2803 raft_consensus.cc:482] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:41:42.947937 2803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:42.948378 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:41:42.948976 2803 raft_consensus.cc:504] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaaddde79524f9389c8e7ef245ac098" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41111 } }
I20260810 22:41:42.949335 2803 leader_election.cc:302] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edaaddde79524f9389c8e7ef245ac098; no voters:
I20260810 22:41:42.949662 2803 leader_election.cc:288] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:41:42.949797 2808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:41:42.950019 2808 raft_consensus.cc:686] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [term 1 LEADER]: Becoming Leader. State: Replica: edaaddde79524f9389c8e7ef245ac098, State: Running, Role: LEADER
I20260810 22:41:42.950389 2808 consensus_queue.cc:237] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaaddde79524f9389c8e7ef245ac098" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41111 } }
I20260810 22:41:42.950848 2803 sys_catalog.cc:564] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:42.951941 2810 sys_catalog.cc:455] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edaaddde79524f9389c8e7ef245ac098. Latest consensus state: current_term: 1 leader_uuid: "edaaddde79524f9389c8e7ef245ac098" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaaddde79524f9389c8e7ef245ac098" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41111 } } }
I20260810 22:41:42.952059 2810 sys_catalog.cc:458] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:42.952481 2819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:41:42.952517 2809 sys_catalog.cc:455] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edaaddde79524f9389c8e7ef245ac098" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edaaddde79524f9389c8e7ef245ac098" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41111 } } }
I20260810 22:41:42.953128 2809 sys_catalog.cc:458] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098 [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:42.955605 2819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:41:42.960567 2819 catalog_manager.cc:1347] Generated new cluster ID: fcd3ae3409b1499ea637263c95e9d830
I20260810 22:41:42.960626 2819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:41:42.967694 2819 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:41:42.968541 2819 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:41:42.977025 2819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edaaddde79524f9389c8e7ef245ac098: Generated new TSK 0
I20260810 22:41:42.977640 2819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:41:43.004676 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2730
2026-08-10T22:41:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1494 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:41:43 dist-test-slave-1x4g krb5kdc[2833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:41:43 dist-test-slave-1x4g krb5kdc[2833](info): set up 2 sockets
Aug 10 22:41:43 dist-test-slave-1x4g krb5kdc[2833](info): commencing operation
krb5kdc: starting...
W20260810 22:41:44.090615 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:41:44 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:41:44.213547 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39375
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:39375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:44.323662 2847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:44.323976 2847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:44.324054 2847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:44.328469 2847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:44.328543 2847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:44.328577 2847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:44.328611 2847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:44.328644 2847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:44.333643 2847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:39375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:44.334623 2847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:44.335598 2847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:44.343927 2854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:44.343904 2847 server_base.cc:1034] running on GCE node
W20260810 22:41:44.344010 2855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:44.343890 2857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:44.344815 2847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:44.346036 2847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:44.347198 2847 hybrid_clock.cc:648] HybridClock initialized: now 1786401704347182 us; error 29 us; skew 500 ppm
Aug 10 22:41:44 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:44.349885 2847 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:41:44.350719 2847 webserver.cc:460] Webserver started at http://127.19.202.126:35565/ using document root <none> and password file <none>
I20260810 22:41:44.351194 2847 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:44.351251 2847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:44.351425 2847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:44.353396 2847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "73774eb8341240a79e8f700077e03ddd"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "4826e79a6d9f7e46647a502f4f8412db"
server_key_iv: "e91e1cb0bc39176bc1ff7947e7866c27"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.353917 2847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "73774eb8341240a79e8f700077e03ddd"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "4826e79a6d9f7e46647a502f4f8412db"
server_key_iv: "e91e1cb0bc39176bc1ff7947e7866c27"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.357442 2847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:41:44.359602 2864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:44.360716 2847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:41:44.360881 2847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "73774eb8341240a79e8f700077e03ddd"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "4826e79a6d9f7e46647a502f4f8412db"
server_key_iv: "e91e1cb0bc39176bc1ff7947e7866c27"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.360972 2847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:44.378394 2847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:44.381184 2847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:44.381331 2847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:44.389398 2847 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:39375
I20260810 22:41:44.389739 2922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:39375 every 8 connection(s)
I20260810 22:41:44.390642 2847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:41:44.393502 2923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:44.399099 2923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Bootstrap starting.
I20260810 22:41:44.400280 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2847
I20260810 22:41:44.400449 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:41:44.400815 20265 external_mini_cluster.cc:1428] Setting key 620ccdb047b5546c4e507a0565ae38f1
I20260810 22:41:44.402112 2923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:44.402928 2923 log.cc:824] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:44.405017 2923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: No bootstrap required, opened a new log
I20260810 22:41:44.408329 2923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:44.408578 2923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:44.408675 2923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73774eb8341240a79e8f700077e03ddd, State: Initialized, Role: FOLLOWER
I20260810 22:41:44.409250 2923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:44.409464 2923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:41:44.409552 2923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:41:44.409665 2923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 0 FOLLOWER]: Advancing to term 1
Aug 10 22:41:44 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:44.410843 2923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:44.411211 2923 leader_election.cc:302] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73774eb8341240a79e8f700077e03ddd; no voters:
I20260810 22:41:44.411592 2923 leader_election.cc:288] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:41:44.411674 2928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:41:44.411939 2928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 LEADER]: Becoming Leader. State: Replica: 73774eb8341240a79e8f700077e03ddd, State: Running, Role: LEADER
I20260810 22:41:44.412299 2928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:44.413046 2923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:44.413609 2930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73774eb8341240a79e8f700077e03ddd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } } }
I20260810 22:41:44.414476 2930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:44.414358 2929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73774eb8341240a79e8f700077e03ddd. Latest consensus state: current_term: 1 leader_uuid: "73774eb8341240a79e8f700077e03ddd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } } }
I20260810 22:41:44.414801 2929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:44.415501 2938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:41:44.416601 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:44.401988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36024 (local address 127.19.202.126:39375)
0810 22:41:44.402474 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:44.402494 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:44.402530 (+ 36us) server_negotiation.cc:407] Connection header received
0810 22:41:44.403231 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:44.403249 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:44.403678 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:44.404070 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:44.404947 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.405961 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.406826 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.407295 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.407475 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:44.410397 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:44.410402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:44.410415 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:44.410441 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:44.412508 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:44.413108 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:44.413114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:44.413120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:44.413281 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:44.413603 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:44.413607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:44.413609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:44.414214 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:44.414415 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:44.415285 (+ 870us) server_negotiation.cc:299] Negotiation successful
0810 22:41:44.415605 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":151,"threads_started":1}
I20260810 22:41:44.418463 2938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:41:44.424199 2938 catalog_manager.cc:1347] Generated new cluster ID: dd95637a32284e74b7d6bc2cf8984257
I20260810 22:41:44.424270 2938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:41:44.440734 2938 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:41:44.442018 2938 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:41:44.449851 2938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Generated new TSK 0
I20260810 22:41:44.450595 2938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:41:44.485862 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39375
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:44.597334 2950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:44.597564 2950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:44.597604 2950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:44.601616 2950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:44.601666 2950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:44.601754 2950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:41:44.606781 2950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:44.607777 2950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:44.608731 2950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:44.618402 2957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:44.618417 2956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:44.618412 2959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:44.618633 2950 server_base.cc:1034] running on GCE node
I20260810 22:41:44.619076 2950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:44.619695 2950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:44.620898 2950 hybrid_clock.cc:648] HybridClock initialized: now 1786401704620863 us; error 41 us; skew 500 ppm
Aug 10 22:41:44 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:44.623706 2950 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:41:44.624603 2950 webserver.cc:460] Webserver started at http://127.19.202.65:45607/ using document root <none> and password file <none>
I20260810 22:41:44.625218 2950 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:44.625280 2950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:44.625540 2950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:44.627679 2950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "dd70afdba0c14e53b4ddff55d795db15"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "1d331bfc27f06bc43ee6d425ac48acb2"
server_key_iv: "7f9f249bf10dbd71d81bfb3f37040456"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.628248 2950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "dd70afdba0c14e53b4ddff55d795db15"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "1d331bfc27f06bc43ee6d425ac48acb2"
server_key_iv: "7f9f249bf10dbd71d81bfb3f37040456"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.632041 2950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 22:41:44.634456 2966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:44.635744 2950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:41:44.635916 2950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "dd70afdba0c14e53b4ddff55d795db15"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "1d331bfc27f06bc43ee6d425ac48acb2"
server_key_iv: "7f9f249bf10dbd71d81bfb3f37040456"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.636047 2950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:44.653822 2950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:44.657169 2950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:44.657390 2950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:44.657982 2950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:44.658798 2950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:44.658885 2950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:44.658983 2950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:44.659036 2950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:44.668903 2950 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:41525
I20260810 22:41:44.668943 3091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:41525 every 8 connection(s)
I20260810 22:41:44.669771 2950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:41:44.672169 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 2950
I20260810 22:41:44.672310 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:41:44.672647 20265 external_mini_cluster.cc:1428] Setting key 371931d60dda41ee14ccfe0f86628698
Aug 10 22:41:44 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:41:44.684206 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:44.671500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:55701 (local address 127.19.202.126:39375)
0810 22:41:44.671651 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:44.671655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:44.672557 (+ 902us) server_negotiation.cc:407] Connection header received
0810 22:41:44.673997 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:44.674000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:44.674054 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:44.674141 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:44.675295 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.675830 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.677099 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.677526 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.677651 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:44.680500 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:44.680503 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:44.680505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:44.680540 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:44.682047 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:44.682696 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:44.682698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:44.682699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:44.682767 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:44.683156 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:44.683159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:44.683160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:44.683339 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:44.683416 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:44.683892 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 22:41:44.683994 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 22:41:44.685078 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:44.671918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:55701)
0810 22:41:44.672380 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:44.672443 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:44.673788 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:44.674333 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:44.674340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:44.674683 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:44.675139 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:44.675152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.676000 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.676006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:44.676979 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:44.676990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.677672 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.677675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:44.677825 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:44.678394 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:44.678409 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:44.680330 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:44.682173 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:44.682183 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:44.682198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:44.682579 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:44.682860 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:44.682863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:44.682865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:44.683047 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:44.683448 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:44.683453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:44.683661 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 22:41:44.683834 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 22:41:44.684076 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":143,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:41:44.686357 3092 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39375
I20260810 22:41:44.686622 3092 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:44.687088 3092 heartbeater.cc:511] Master 127.19.202.126:39375 requested a full tablet report, sending...
I20260810 22:41:44.688642 2882 ts_manager.cc:194] Registered new tserver with Master: dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525)
I20260810 22:41:44.689822 2882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:55701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:41:44.704026 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39375
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:44.813717 3098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:44.813951 3098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:44.814033 3098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:44.818243 3098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:44.818295 3098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:44.818387 3098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:41:44.823342 3098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:44.824308 3098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:44.825377 3098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:44.838073 3104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:44.838184 3106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:44.838299 3103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:44.838577 3098 server_base.cc:1034] running on GCE node
I20260810 22:41:44.839217 3098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:44.840116 3098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:44.841363 3098 hybrid_clock.cc:648] HybridClock initialized: now 1786401704841328 us; error 49 us; skew 500 ppm
Aug 10 22:41:44 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:44.845398 3098 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:41:44.846593 3098 webserver.cc:460] Webserver started at http://127.19.202.66:36329/ using document root <none> and password file <none>
I20260810 22:41:44.847368 3098 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:44.847455 3098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:44.847726 3098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:44.850739 3098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "ccb43aea1d0c42f4a5655b52a9509ce6"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "6a6b537490a63eee23f71a5937e92826"
server_key_iv: "1f2081fc4894ed5e89dbb833b47137cc"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.851558 3098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "ccb43aea1d0c42f4a5655b52a9509ce6"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "6a6b537490a63eee23f71a5937e92826"
server_key_iv: "1f2081fc4894ed5e89dbb833b47137cc"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.856544 3098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 22:41:44.859156 3113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:44.860477 3098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:41:44.860601 3098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "ccb43aea1d0c42f4a5655b52a9509ce6"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "6a6b537490a63eee23f71a5937e92826"
server_key_iv: "1f2081fc4894ed5e89dbb833b47137cc"
server_key_version: "encryptionkey@0"
I20260810 22:41:44.860713 3098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:44.882156 3098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:44.885471 3098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:44.885694 3098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:44.886260 3098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:44.887048 3098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:44.887094 3098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:44.887159 3098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:44.887199 3098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:44.895884 3098 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:39033
I20260810 22:41:44.895910 3235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:39033 every 8 connection(s)
I20260810 22:41:44.896739 3098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:41:44.900368 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 3098
I20260810 22:41:44.900489 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:41:44.900756 20265 external_mini_cluster.cc:1428] Setting key 4041795eba8c14c409dd30731dc3020c
Aug 10 22:41:44 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401704, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:44.910537 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:44.898482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:49091 (local address 127.19.202.126:39375)
0810 22:41:44.898617 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:44.898621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:44.899354 (+ 733us) server_negotiation.cc:407] Connection header received
0810 22:41:44.900829 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:44.900832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:44.900891 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:44.901009 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:44.902358 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.902866 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.904011 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.904235 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.904347 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:44.906986 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:44.906989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:44.906991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:44.907013 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:44.908442 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:44.909037 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:44.909041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:44.909042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:44.909120 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:44.909477 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:44.909482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:44.909484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:44.909664 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:44.909769 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:44.910254 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 22:41:44.910372 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:41:44.911346 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:44.898800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:49091)
0810 22:41:44.899216 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:44.899254 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:44.900610 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:44.901180 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:44.901191 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:44.901657 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:44.902213 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:44.902229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.902999 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.903002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:44.903879 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:44.903888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:44.904395 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:44.904399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:44.904577 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:44.905155 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:44.905170 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:44.906778 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:44.908570 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:44.908576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:44.908588 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:44.908931 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:44.909212 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:44.909215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:44.909217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:44.909354 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:44.909768 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:44.909775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:44.910008 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 22:41:44.910207 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 22:41:44.910518 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260810 22:41:44.912461 3236 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39375
I20260810 22:41:44.912707 3236 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:44.913235 3236 heartbeater.cc:511] Master 127.19.202.126:39375 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
I20260810 22:41:44.914505 2882 ts_manager.cc:194] Registered new tserver with Master: ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033)
I20260810 22:41:44.915364 2882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:49091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:41:44.931555 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39375
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:45.043278 3242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:45.043541 3242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:45.043617 3242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:45.047549 3242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:45.047601 3242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:45.047706 3242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:41:45.052743 3242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:45.053784 3242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:45.054778 3242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:45.064302 3249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:45.064482 3242 server_base.cc:1034] running on GCE node
W20260810 22:41:45.064304 3248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:45.064308 3251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:45.065068 3242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:45.065704 3242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:45.066866 3242 hybrid_clock.cc:648] HybridClock initialized: now 1786401705066851 us; error 26 us; skew 500 ppm
Aug 10 22:41:45 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:45.069599 3242 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:41:45.070372 3242 webserver.cc:460] Webserver started at http://127.19.202.67:41339/ using document root <none> and password file <none>
I20260810 22:41:45.070868 3242 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:45.070947 3242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:45.071101 3242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:41:45.073107 3242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "04a18f23d5364dee970a4b2d2446bd74"
format_stamp: "Formatted at 2026-08-10 22:41:45 on dist-test-slave-1x4g"
server_key: "b73021a7a18e0c7b864e639c5aa19b5b"
server_key_iv: "098ea57f80886e02857a1b0bd5c2fa19"
server_key_version: "encryptionkey@0"
I20260810 22:41:45.073611 3242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "04a18f23d5364dee970a4b2d2446bd74"
format_stamp: "Formatted at 2026-08-10 22:41:45 on dist-test-slave-1x4g"
server_key: "b73021a7a18e0c7b864e639c5aa19b5b"
server_key_iv: "098ea57f80886e02857a1b0bd5c2fa19"
server_key_version: "encryptionkey@0"
I20260810 22:41:45.077291 3242 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:41:45.079630 3259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:45.080950 3242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:41:45.081085 3242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "04a18f23d5364dee970a4b2d2446bd74"
format_stamp: "Formatted at 2026-08-10 22:41:45 on dist-test-slave-1x4g"
server_key: "b73021a7a18e0c7b864e639c5aa19b5b"
server_key_iv: "098ea57f80886e02857a1b0bd5c2fa19"
server_key_version: "encryptionkey@0"
I20260810 22:41:45.081184 3242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:45.107199 3242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:45.110556 3242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:45.110767 3242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:45.111384 3242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:45.112232 3242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:45.112278 3242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:45.112346 3242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:45.112396 3242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:45.121399 3242 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:39121
I20260810 22:41:45.121420 3377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:39121 every 8 connection(s)
I20260810 22:41:45.122246 3242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:41:45.128005 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 3242
I20260810 22:41:45.128127 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:41:45.128400 20265 external_mini_cluster.cc:1428] Setting key 9d1a0b8d8ba42651ac6449b6708bb171
Aug 10 22:41:45 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:45.136452 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.124037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:45071 (local address 127.19.202.126:39375)
0810 22:41:45.124170 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.124173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.125003 (+ 830us) server_negotiation.cc:407] Connection header received
0810 22:41:45.126380 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.126383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.126441 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.126563 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:45.127725 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.128523 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.129939 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.130232 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.130339 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.133031 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:45.133034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:45.133037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:45.133061 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:45.134576 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.135100 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.135103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.135104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.135181 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.135486 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.135488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.135489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.135675 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:45.135750 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.136224 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.136327 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:41:45.137259 3380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.124330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:45071)
0810 22:41:45.124823 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.124886 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.126170 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.126741 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.126748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.127097 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:45.127569 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.127581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.128681 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.128684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.129855 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.129864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.130414 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.130421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.130606 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.131235 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:45.131251 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:45.132880 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:45.134682 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:45.134687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:45.134698 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:45.134973 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:45.135258 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:45.135261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:45.135262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:45.135396 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:45.135762 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:45.135767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:45.136007 (+ 240us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.136154 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.136387 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
I20260810 22:41:45.138401 3378 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39375
I20260810 22:41:45.138628 3378 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:45.139114 3378 heartbeater.cc:511] Master 127.19.202.126:39375 requested a full tablet report, sending...
I20260810 22:41:45.140172 2882 ts_manager.cc:194] Registered new tserver with Master: 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121)
I20260810 22:41:45.140749 2882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:45071
I20260810 22:41:45.142421 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:41:45.151304 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.144422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36038 (local address 127.19.202.126:39375)
0810 22:41:45.144542 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.144546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.144705 (+ 159us) server_negotiation.cc:407] Connection header received
0810 22:41:45.144826 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.144829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.144873 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.144988 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:45.145700 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.146150 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.146906 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.147124 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.147197 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.148585 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:45.148589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:45.148591 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:45.148611 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:45.149902 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.150304 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.150307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.150308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.150379 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.150622 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.150624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.150625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.150829 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:45.150926 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.151110 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.151197 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:41:45.153961 2882 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:41:45.156999 2882 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:41:45.172096 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.166050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.19.202.67:39121)
0810 22:41:45.167624 (+ 1574us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.167632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.167649 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:41:45.167706 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.167710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.167874 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.168016 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:45.168403 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.169491 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.170883 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.171373 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.171491 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.171638 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.171811 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.171945 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1491,"thread_start_us":98,"threads_started":1}
I20260810 22:41:45.172132 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.165717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39121 (local address 127.0.0.1:35174)
0810 22:41:45.166259 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.166274 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.166358 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.168172 (+ 1814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.168176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.168204 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:45.168289 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.168294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.169644 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.169648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.170753 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.170760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.171493 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.171496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.171614 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.171651 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.171765 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.171919 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":6,"thread_start_us":96,"threads_started":1}
I20260810 22:41:45.172169 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.165484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41525 (local address 127.0.0.1:39626)
0810 22:41:45.166309 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.166330 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.166427 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.167350 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.167357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.167399 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:45.167760 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.167767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.169052 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.169059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.170290 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.170300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.171634 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.171638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.171730 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.171754 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.171864 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.171959 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":159,"spinlock_wait_cycles":8576,"thread_start_us":129,"threads_started":1}
I20260810 22:41:45.172843 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.166126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39626 (local address 127.19.202.65:41525)
0810 22:41:45.166944 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.166958 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.166977 (+ 19us) server_negotiation.cc:407] Connection header received
0810 22:41:45.167041 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.167046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.167182 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.167326 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:45.167942 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.168706 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.170439 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.171277 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.171394 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.171562 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.172515 (+ 953us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.172640 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":108,"threads_started":1}
I20260810 22:41:45.173144 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.166478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.19.202.66:39033)
0810 22:41:45.166839 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.166850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.166876 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:41:45.166968 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.166973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.167168 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.167326 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:45.167901 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.169064 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.170614 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.171529 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.171638 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.171767 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.172667 (+ 900us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.172763 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":169,"threads_started":1}
I20260810 22:41:45.173460 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.165255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39033 (local address 127.0.0.1:56164)
0810 22:41:45.165617 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.165671 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.165802 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.167350 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.167357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.167397 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:45.167759 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.167767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.169202 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.169205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.170481 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.170492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.171849 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.171856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.171973 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.172005 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.172082 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.172846 (+ 764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":117,"spinlock_wait_cycles":15744,"thread_start_us":79,"threads_started":1}
I20260810 22:41:45.175850 3020 tablet_service.cc:1467] Processing CreateTablet for tablet d033bcd53e1744bc8ba0a58992b88ca9 (DEFAULT_TABLE table=test-table [id=495fe53bd2b54f00bbc464ccd2b8c4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:45.175884 3308 tablet_service.cc:1467] Processing CreateTablet for tablet d033bcd53e1744bc8ba0a58992b88ca9 (DEFAULT_TABLE table=test-table [id=495fe53bd2b54f00bbc464ccd2b8c4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:45.177052 3308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d033bcd53e1744bc8ba0a58992b88ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:45.177053 3020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d033bcd53e1744bc8ba0a58992b88ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:45.176913 3164 tablet_service.cc:1467] Processing CreateTablet for tablet d033bcd53e1744bc8ba0a58992b88ca9 (DEFAULT_TABLE table=test-table [id=495fe53bd2b54f00bbc464ccd2b8c4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:45.177812 3164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d033bcd53e1744bc8ba0a58992b88ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:45.183053 3399 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Bootstrap starting.
I20260810 22:41:45.184456 3401 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Bootstrap starting.
I20260810 22:41:45.185060 3400 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Bootstrap starting.
I20260810 22:41:45.185441 3399 tablet_bootstrap.cc:654] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:45.186827 3399 log.cc:824] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:45.187054 3400 tablet_bootstrap.cc:654] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:45.187637 3401 tablet_bootstrap.cc:654] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:45.188257 3401 log.cc:824] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:45.188295 3400 log.cc:824] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:45.189057 3399 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: No bootstrap required, opened a new log
I20260810 22:41:45.189255 3399 ts_tablet_manager.cc:1397] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260810 22:41:45.190014 3400 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: No bootstrap required, opened a new log
I20260810 22:41:45.190047 3401 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: No bootstrap required, opened a new log
I20260810 22:41:45.190193 3400 ts_tablet_manager.cc:1397] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260810 22:41:45.190201 3401 ts_tablet_manager.cc:1397] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:41:45.192847 3400 raft_consensus.cc:348] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.193116 3400 raft_consensus.cc:374] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:45.192945 3399 raft_consensus.cc:348] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.193181 3400 raft_consensus.cc:729] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a18f23d5364dee970a4b2d2446bd74, State: Initialized, Role: FOLLOWER
I20260810 22:41:45.193187 3399 raft_consensus.cc:374] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:45.193234 3399 raft_consensus.cc:729] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd70afdba0c14e53b4ddff55d795db15, State: Initialized, Role: FOLLOWER
I20260810 22:41:45.193670 3399 consensus_queue.cc:260] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.193656 3400 consensus_queue.cc:260] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.193871 3401 raft_consensus.cc:348] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.194099 3401 raft_consensus.cc:374] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:45.194147 3401 raft_consensus.cc:729] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb43aea1d0c42f4a5655b52a9509ce6, State: Initialized, Role: FOLLOWER
I20260810 22:41:45.194550 3401 consensus_queue.cc:260] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.194643 3399 ts_tablet_manager.cc:1428] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260810 22:41:45.194654 3400 ts_tablet_manager.cc:1428] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260810 22:41:45.194998 3092 heartbeater.cc:503] Master 127.19.202.126:39375 was elected leader, sending a full tablet report...
I20260810 22:41:45.195051 3378 heartbeater.cc:503] Master 127.19.202.126:39375 was elected leader, sending a full tablet report...
I20260810 22:41:45.195359 3401 ts_tablet_manager.cc:1428] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:41:45.195703 3236 heartbeater.cc:503] Master 127.19.202.126:39375 was elected leader, sending a full tablet report...
I20260810 22:41:45.323431 3405 raft_consensus.cc:482] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:45.323758 3405 raft_consensus.cc:504] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.325006 3405 leader_election.cc:288] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033), 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121)
I20260810 22:41:45.329154 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.325259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39033 (local address 127.19.202.65:59927)
0810 22:41:45.325879 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.325893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.325988 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.326268 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.326270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.326284 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:45.326429 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.326437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.327148 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.327151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.327912 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.327920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.328588 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.328591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.328769 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.328836 (+ 67us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.328908 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.328961 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":358,"thread_start_us":110,"threads_started":1}
I20260810 22:41:45.329241 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.325394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:33945 (local address 127.19.202.67:39121)
0810 22:41:45.325552 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.325557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.325748 (+ 191us) server_negotiation.cc:407] Connection header received
0810 22:41:45.325943 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.325950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.326002 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.326075 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:45.326392 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.326991 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.328033 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.328441 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.328588 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.328660 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.329040 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.329112 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 22:41:45.329241 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.325359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:59927 (local address 127.19.202.66:39033)
0810 22:41:45.325533 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.325538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.325975 (+ 437us) server_negotiation.cc:407] Connection header received
0810 22:41:45.326102 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.326105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.326161 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.326237 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:45.326552 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.327014 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.328036 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.328440 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.328588 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.328660 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.329023 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.329100 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 22:41:45.329319 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.325259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39121 (local address 127.19.202.65:33945)
0810 22:41:45.325624 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.325639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.325801 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.326104 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.326107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.326124 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:45.326274 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.326282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.327103 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.327106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.327907 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.327916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.328588 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.328592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.328789 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.328807 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.329029 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.329100 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":56,"thread_start_us":149,"threads_started":1}
I20260810 22:41:45.329960 3330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d033bcd53e1744bc8ba0a58992b88ca9" candidate_uuid: "dd70afdba0c14e53b4ddff55d795db15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a18f23d5364dee970a4b2d2446bd74" is_pre_election: true
I20260810 22:41:45.329959 3185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d033bcd53e1744bc8ba0a58992b88ca9" candidate_uuid: "dd70afdba0c14e53b4ddff55d795db15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" is_pre_election: true
I20260810 22:41:45.330248 3185 raft_consensus.cc:2454] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd70afdba0c14e53b4ddff55d795db15 in term 0.
I20260810 22:41:45.330253 3330 raft_consensus.cc:2454] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd70afdba0c14e53b4ddff55d795db15 in term 0.
I20260810 22:41:45.330710 2968 leader_election.cc:302] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccb43aea1d0c42f4a5655b52a9509ce6, dd70afdba0c14e53b4ddff55d795db15; no voters:
I20260810 22:41:45.331001 3405 raft_consensus.cc:2781] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:41:45.331103 3405 raft_consensus.cc:482] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:41:45.331158 3405 raft_consensus.cc:3037] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:45.332139 3405 raft_consensus.cc:504] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.332463 3405 leader_election.cc:288] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 1 election: Requested vote from peers ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033), 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121)
I20260810 22:41:45.332753 3185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d033bcd53e1744bc8ba0a58992b88ca9" candidate_uuid: "dd70afdba0c14e53b4ddff55d795db15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6"
I20260810 22:41:45.332917 3330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d033bcd53e1744bc8ba0a58992b88ca9" candidate_uuid: "dd70afdba0c14e53b4ddff55d795db15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a18f23d5364dee970a4b2d2446bd74"
I20260810 22:41:45.332907 3185 raft_consensus.cc:3037] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:45.333062 3330 raft_consensus.cc:3037] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:45.334084 3185 raft_consensus.cc:2454] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd70afdba0c14e53b4ddff55d795db15 in term 1.
I20260810 22:41:45.334203 3330 raft_consensus.cc:2454] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd70afdba0c14e53b4ddff55d795db15 in term 1.
I20260810 22:41:45.334357 2968 leader_election.cc:302] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccb43aea1d0c42f4a5655b52a9509ce6, dd70afdba0c14e53b4ddff55d795db15; no voters:
I20260810 22:41:45.334533 3405 raft_consensus.cc:2781] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:41:45.334771 3405 raft_consensus.cc:686] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 LEADER]: Becoming Leader. State: Replica: dd70afdba0c14e53b4ddff55d795db15, State: Running, Role: LEADER
I20260810 22:41:45.335131 3405 consensus_queue.cc:237] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.338403 2882 catalog_manager.cc:5495] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "dd70afdba0c14e53b4ddff55d795db15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } health_report { overall_health: UNKNOWN } } }
I20260810 22:41:45.359242 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.355602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.19.202.65:41525)
0810 22:41:45.355743 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.355747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.355760 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:41:45.355944 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.355946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.355996 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.356111 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:41:45.356355 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.356968 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.357835 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.358056 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.358161 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.358484 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:41:45.358871 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:41:45.358998 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.359060 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.359113 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:41:45.364756 3185 raft_consensus.cc:1261] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Refusing update from remote peer dd70afdba0c14e53b4ddff55d795db15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:45.364756 3330 raft_consensus.cc:1261] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 1 FOLLOWER]: Refusing update from remote peer dd70afdba0c14e53b4ddff55d795db15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:45.365586 3410 consensus_queue.cc:1035] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:41:45.365782 3405 consensus_queue.cc:1035] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 22:41:45.373100 3379 tablet.cc:2369] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:41:45.373571 3414 mvcc.cc:205] Tried to move back new op lower bound from 7317101385166471168 to 7317101385055629312. Current Snapshot: MvccSnapshot[applied={T|T < 7317101385166471168}]
I20260810 22:41:45.373611 3416 mvcc.cc:205] Tried to move back new op lower bound from 7317101385166471168 to 7317101385055629312. Current Snapshot: MvccSnapshot[applied={T|T < 7317101385166471168}]
I20260810 22:41:45.374709 3417 mvcc.cc:205] Tried to move back new op lower bound from 7317101385166471168 to 7317101385055629312. Current Snapshot: MvccSnapshot[applied={T|T < 7317101385166471168}]
I20260810 22:41:45.382766 2881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:41:45.382925 2881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:41:45.385376 2881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Sending DeleteTablet for 3 replicas of tablet d033bcd53e1744bc8ba0a58992b88ca9
I20260810 22:41:45.386137 3164 tablet_service.cc:1514] Processing DeleteTablet for tablet d033bcd53e1744bc8ba0a58992b88ca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:45 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:56164
I20260810 22:41:45.386168 3020 tablet_service.cc:1514] Processing DeleteTablet for tablet d033bcd53e1744bc8ba0a58992b88ca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:45 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:39626
I20260810 22:41:45.386595 3426 tablet_replica.cc:330] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: stopping tablet replica
I20260810 22:41:45.386612 3427 tablet_replica.cc:330] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: stopping tablet replica
I20260810 22:41:45.386741 3308 tablet_service.cc:1514] Processing DeleteTablet for tablet d033bcd53e1744bc8ba0a58992b88ca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:45 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:35174
I20260810 22:41:45.386909 3426 raft_consensus.cc:2229] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:41:45.387087 3427 raft_consensus.cc:2229] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:41:45.387221 3426 raft_consensus.cc:2258] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:41:45.387519 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2950
I20260810 22:41:45.391654 3428 tablet_replica.cc:330] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: stopping tablet replica
I20260810 22:41:45.391924 3428 raft_consensus.cc:2229] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:41:45.392207 3428 raft_consensus.cc:2258] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:41:45.393596 3428 ts_tablet_manager.cc:1905] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:41:45.394702 3426 ts_tablet_manager.cc:1905] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:41:45.396364 3428 ts_tablet_manager.cc:1918] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:41:45.396468 3428 log.cc:1196] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/d033bcd53e1744bc8ba0a58992b88ca9
I20260810 22:41:45.396875 3428 ts_tablet_manager.cc:1939] T d033bcd53e1744bc8ba0a58992b88ca9 P 04a18f23d5364dee970a4b2d2446bd74: Deleting consensus metadata
I20260810 22:41:45.397578 3426 ts_tablet_manager.cc:1918] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:41:45.397679 3426 log.cc:1196] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/d033bcd53e1744bc8ba0a58992b88ca9
I20260810 22:41:45.398010 3426 ts_tablet_manager.cc:1939] T d033bcd53e1744bc8ba0a58992b88ca9 P ccb43aea1d0c42f4a5655b52a9509ce6: Deleting consensus metadata
I20260810 22:41:45.398161 2865 catalog_manager.cc:4841] TS 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121): tablet d033bcd53e1744bc8ba0a58992b88ca9 (table test-table [id=495fe53bd2b54f00bbc464ccd2b8c4be]) successfully deleted
I20260810 22:41:45.399102 2866 catalog_manager.cc:4841] TS ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033): tablet d033bcd53e1744bc8ba0a58992b88ca9 (table test-table [id=495fe53bd2b54f00bbc464ccd2b8c4be]) successfully deleted
W20260810 22:41:45.404610 2868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:41525 (error 108)
I20260810 22:41:45.404762 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 3098
I20260810 22:41:45.408130 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.407693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41525 (local address 127.0.0.1:39654)
0810 22:41:45.407972 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.408040 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:41525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
W20260810 22:41:45.412091 2868 catalog_manager.cc:4584] TS dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d033bcd53e1744bc8ba0a58992b88ca9: Network error: Client connection negotiation failed: client connection to 127.19.202.65:41525: connect: Connection refused (error 111)
I20260810 22:41:45.418565 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 3242
I20260810 22:41:45.430069 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 2847
I20260810 22:41:45.438400 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39375
--webserver_interface=127.19.202.126
--webserver_port=35565
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:39375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:45.548986 3430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:45.549279 3430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:45.549357 3430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:45.553310 3430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:45.553391 3430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:45.553431 3430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:45.553491 3430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:45.553534 3430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:45.558698 3430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:39375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=35565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:45.559659 3430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:45.560626 3430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:45.574232 3437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:45.574232 3439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:45.574280 3430 server_base.cc:1034] running on GCE node
W20260810 22:41:45.574232 3436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:45.575127 3430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:45.576188 3430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:45.577373 3430 hybrid_clock.cc:648] HybridClock initialized: now 1786401705577356 us; error 35 us; skew 500 ppm
Aug 10 22:41:45 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:45.580046 3430 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:41:45.581007 3430 webserver.cc:460] Webserver started at http://127.19.202.126:35565/ using document root <none> and password file <none>
I20260810 22:41:45.581562 3430 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:45.581632 3430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:45.585359 3430 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 22:41:45.587702 3446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:45.588975 3430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:41:45.589097 3430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "73774eb8341240a79e8f700077e03ddd"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "4826e79a6d9f7e46647a502f4f8412db"
server_key_iv: "e91e1cb0bc39176bc1ff7947e7866c27"
server_key_version: "encryptionkey@0"
I20260810 22:41:45.589682 3430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:45.602029 3430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:45.604758 3430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:45.604993 3430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:45.612061 3430 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:39375
I20260810 22:41:45.612361 3505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:39375 every 8 connection(s)
I20260810 22:41:45.613204 3430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:41:45.614611 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 3430
I20260810 22:41:45.615049 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:41525
--local_ip_for_outbound_sockets=127.19.202.65
--tserver_master_addrs=127.19.202.126:39375
--webserver_port=45607
--webserver_interface=127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:45.617815 3506 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:41:45.619426 3506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Bootstrap starting.
I20260810 22:41:45.637370 3506 log.cc:824] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:45.646086 3506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:41:45.646471 3506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Bootstrap complete.
I20260810 22:41:45.650630 3506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:45.651024 3506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73774eb8341240a79e8f700077e03ddd, State: Initialized, Role: FOLLOWER
I20260810 22:41:45.651497 3506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:45.651652 3506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:41:45.651736 3506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:41:45.651854 3506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:41:45.653720 3506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:45.654078 3506 leader_election.cc:302] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73774eb8341240a79e8f700077e03ddd; no voters:
I20260810 22:41:45.654439 3506 leader_election.cc:288] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 22:41:45.654488 3510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 FOLLOWER]: Leader election won for term 2
I20260810 22:41:45.654685 3510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 LEADER]: Becoming Leader. State: Replica: 73774eb8341240a79e8f700077e03ddd, State: Running, Role: LEADER
I20260810 22:41:45.654976 3510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:45.655638 3506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:45.655870 3512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73774eb8341240a79e8f700077e03ddd. Latest consensus state: current_term: 2 leader_uuid: "73774eb8341240a79e8f700077e03ddd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } } }
I20260810 22:41:45.655926 3511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "73774eb8341240a79e8f700077e03ddd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } } }
I20260810 22:41:45.656049 3511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:45.656000 3512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:45.656394 3514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:41:45.661402 3514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:41:45.662072 3514 catalog_manager.cc:1259] Loaded cluster ID: dd95637a32284e74b7d6bc2cf8984257
I20260810 22:41:45.662146 3514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:41:45.663225 3514 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 22:41:45.665963 3514 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 22:41:45.670037 3514 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:41:45.670756 3514 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Loaded TSK: 0
I20260810 22:41:45.671689 3514 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 22:41:45.748910 3508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:45.749140 3508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:45.749191 3508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:45.753072 3508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:45.753120 3508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:45.753216 3508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:41:45.758659 3508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:41525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=45607
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:45.759616 3508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:45.760614 3508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:45.771167 3534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:45.771257 3537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:45.771247 3535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:45.772140 3508 server_base.cc:1034] running on GCE node
I20260810 22:41:45.772564 3508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:45.773269 3508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:45.774454 3508 hybrid_clock.cc:648] HybridClock initialized: now 1786401705774426 us; error 42 us; skew 500 ppm
Aug 10 22:41:45 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:45.777205 3508 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:41:45.778128 3508 webserver.cc:460] Webserver started at http://127.19.202.65:45607/ using document root <none> and password file <none>
I20260810 22:41:45.778678 3508 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:45.778751 3508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:45.782612 3508 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:41:45.784729 3544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:45.786074 3508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:41:45.786167 3508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "dd70afdba0c14e53b4ddff55d795db15"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "1d331bfc27f06bc43ee6d425ac48acb2"
server_key_iv: "7f9f249bf10dbd71d81bfb3f37040456"
server_key_version: "encryptionkey@0"
I20260810 22:41:45.786799 3508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:45.807296 3508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:45.810433 3508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:45.810643 3508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:45.811324 3508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:45.812497 3553 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 22:41:45.816821 3508 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 22:41:45.816875 3508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 22:41:45.816949 3508 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 22:41:45.818588 3508 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 22:41:45.818645 3508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 22:41:45.818904 3553 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Bootstrap starting.
I20260810 22:41:45.829730 3508 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:41525
I20260810 22:41:45.829759 3667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:41525 every 8 connection(s)
I20260810 22:41:45.830915 3508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:41:45.833316 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 3508
I20260810 22:41:45.833863 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:39033
--local_ip_for_outbound_sockets=127.19.202.66
--tserver_master_addrs=127.19.202.126:39375
--webserver_port=36329
--webserver_interface=127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:45.840766 3553 log.cc:824] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Log is configured to *not* fsync() on all Append() calls
Aug 10 22:41:45 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:45.849411 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.833219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:45371)
0810 22:41:45.834336 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.834395 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.835670 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.836891 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.836899 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.837332 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:45.837783 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.837796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.839094 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.839096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.839881 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.839890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.840645 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.840649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.840866 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.841477 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:45.841496 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:45.843296 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:45.846080 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:45.846085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:45.846096 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:45.846436 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:45.846885 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:45.846888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:45.846890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:45.847055 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:45.847795 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:45.847800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:45.848055 (+ 255us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.848224 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.848507 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":167,"threads_started":1}
I20260810 22:41:45.849607 3553 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:41:45.849964 3553 tablet_bootstrap.cc:492] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Bootstrap complete.
I20260810 22:41:45.850419 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.833179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:45371 (local address 127.19.202.126:39375)
0810 22:41:45.833762 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.833782 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.834553 (+ 771us) server_negotiation.cc:407] Connection header received
0810 22:41:45.836152 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.836163 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.836442 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.836831 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:45.838000 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.838926 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.840007 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.840473 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.840677 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.843507 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:45.843517 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:45.843535 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:45.843574 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:45.845942 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.846580 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.846588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.846597 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.846767 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.847173 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.847178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.847182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.847663 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:45.847806 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.848411 (+ 605us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.848726 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":123,"threads_started":1}
I20260810 22:41:45.851271 3553 ts_tablet_manager.cc:1397] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.008s
I20260810 22:41:45.852641 3668 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39375
I20260810 22:41:45.852900 3668 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:45.853421 3668 heartbeater.cc:511] Master 127.19.202.126:39375 requested a full tablet report, sending...
I20260810 22:41:45.853885 3553 raft_consensus.cc:348] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.854372 3553 raft_consensus.cc:729] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd70afdba0c14e53b4ddff55d795db15, State: Initialized, Role: FOLLOWER
I20260810 22:41:45.854823 3553 consensus_queue.cc:260] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } } peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } }
I20260810 22:41:45.855822 3553 ts_tablet_manager.cc:1428] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Time spent starting tablet: real 0.004s user 0.006s sys 0.001s
I20260810 22:41:45.856078 3466 ts_manager.cc:194] Registered new tserver with Master: dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525)
I20260810 22:41:45.860355 3466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:45371
I20260810 22:41:45.862833 3668 heartbeater.cc:503] Master 127.19.202.126:39375 was elected leader, sending a full tablet report...
Aug 10 22:41:45 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:41:45.870458 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.859285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41525 (local address 127.0.0.1:39660)
0810 22:41:45.859637 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:45.859677 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:45.859790 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:45.861170 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:45.861176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:45.861595 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:45.861715 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.861722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.862998 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.863004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.863967 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:45.863979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.864756 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.864759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:45.864913 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.865525 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:45.865537 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:45.866876 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:45.868877 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:45.868885 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:45.868889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:45.869110 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:45.869427 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:45.869429 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:45.869430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:45.869505 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:45.870008 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:45.870012 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:45.870102 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 22:41:45.870205 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 22:41:45.870338 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":110,"threads_started":1}
I20260810 22:41:45.870646 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:45.859341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.19.202.65:41525)
0810 22:41:45.860698 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:45.860707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:45.860725 (+ 18us) server_negotiation.cc:407] Connection header received
0810 22:41:45.860818 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:45.860827 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:45.861020 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:45.861176 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:45.861846 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.862825 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.864116 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:45.864632 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:45.864759 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:45.867009 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:45.867020 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:45.867032 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:45.867056 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:45.868724 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.869218 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.869224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.869226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.869321 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:45.869591 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:45.869594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:45.869598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:45.869869 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:45.870008 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:45.870309 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 22:41:45.870482 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1277,"thread_start_us":107,"threads_started":1}
I20260810 22:41:45.871563 3600 tablet_service.cc:1514] Processing DeleteTablet for tablet d033bcd53e1744bc8ba0a58992b88ca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:45 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:39660
I20260810 22:41:45.872354 3679 tablet_replica.cc:330] stopping tablet replica
I20260810 22:41:45.872649 3679 raft_consensus.cc:2229] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:41:45.872921 3679 raft_consensus.cc:2258] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:41:45.874010 3679 ts_tablet_manager.cc:1905] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:41:45.878372 3679 ts_tablet_manager.cc:1918] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:41:45.878499 3679 log.cc:1196] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/d033bcd53e1744bc8ba0a58992b88ca9
I20260810 22:41:45.878810 3679 ts_tablet_manager.cc:1939] T d033bcd53e1744bc8ba0a58992b88ca9 P dd70afdba0c14e53b4ddff55d795db15: Deleting consensus metadata
I20260810 22:41:45.880601 3449 catalog_manager.cc:4841] TS dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525): tablet d033bcd53e1744bc8ba0a58992b88ca9 (table test-table [id=495fe53bd2b54f00bbc464ccd2b8c4be]) successfully deleted
W20260810 22:41:45.959040 3674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:45.959321 3674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:45.959396 3674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:45.963187 3674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:45.963272 3674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:45.963352 3674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:41:45.968761 3674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:39033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=36329
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:45.969852 3674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:45.970872 3674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:45.980561 3684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:45.980590 3687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:45.980614 3685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:45.980752 3674 server_base.cc:1034] running on GCE node
I20260810 22:41:45.981177 3674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:45.981793 3674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:45.982980 3674 hybrid_clock.cc:648] HybridClock initialized: now 1786401705982970 us; error 33 us; skew 500 ppm
Aug 10 22:41:45 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:45.985781 3674 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:41:45.986611 3674 webserver.cc:460] Webserver started at http://127.19.202.66:36329/ using document root <none> and password file <none>
I20260810 22:41:45.987244 3674 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:45.987352 3674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:45.991170 3674 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:41:45.993247 3694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:45.994450 3674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 22:41:45.994583 3674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "ccb43aea1d0c42f4a5655b52a9509ce6"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "6a6b537490a63eee23f71a5937e92826"
server_key_iv: "1f2081fc4894ed5e89dbb833b47137cc"
server_key_version: "encryptionkey@0"
I20260810 22:41:45.995114 3674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:46.018455 3674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:46.021119 3674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:46.021318 3674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:46.021889 3674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:46.022698 3674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:46.022771 3674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:46.022837 3674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:46.022887 3674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:46.031889 3674 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:39033
I20260810 22:41:46.031939 3817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:39033 every 8 connection(s)
I20260810 22:41:46.032874 3674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:41:46.035746 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 3674
I20260810 22:41:46.036165 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:39121
--local_ip_for_outbound_sockets=127.19.202.67
--tserver_master_addrs=127.19.202.126:39375
--webserver_port=41339
--webserver_interface=127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:41:46 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401705, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:46.046885 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.034624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:44297 (local address 127.19.202.126:39375)
0810 22:41:46.034758 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.034762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.035606 (+ 844us) server_negotiation.cc:407] Connection header received
0810 22:41:46.037213 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.037217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.037279 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.037356 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:46.038470 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.039005 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.040137 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.040391 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.040507 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.043200 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:46.043203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:46.043205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:46.043225 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:46.044838 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:46.045362 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:46.045366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:46.045368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:46.045451 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:46.045789 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:46.045793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:46.045795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:46.045982 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:46.046102 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.046585 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.046716 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:41:46.047624 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.035007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:44297)
0810 22:41:46.035453 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.035493 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.037003 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.037527 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.037534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.037867 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:46.038306 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.038318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.039141 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.039149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.040010 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.040019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.040546 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.040550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.040713 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.041356 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:46.041372 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:46.043032 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:46.044975 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:46.044981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:46.044994 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:46.045256 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:46.045546 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:46.045548 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:46.045550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:46.045693 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:46.046102 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:46.046108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:46.046356 (+ 248us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.046542 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.046837 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
I20260810 22:41:46.048959 3818 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39375
I20260810 22:41:46.049201 3818 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:46.049685 3818 heartbeater.cc:511] Master 127.19.202.126:39375 requested a full tablet report, sending...
I20260810 22:41:46.050774 3466 ts_manager.cc:194] Registered new tserver with Master: ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033)
I20260810 22:41:46.051506 3466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:44297
W20260810 22:41:46.149560 3821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:46.149791 3821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:46.149832 3821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:46.153789 3821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:46.153843 3821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:46.153898 3821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:41:46.159405 3821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:39121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=41339
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:46.160359 3821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:46.161347 3821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:46.170933 3826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:46.171072 3827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:46.170941 3829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:46.171192 3821 server_base.cc:1034] running on GCE node
I20260810 22:41:46.171685 3821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:46.172364 3821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:46.173573 3821 hybrid_clock.cc:648] HybridClock initialized: now 1786401706173551 us; error 47 us; skew 500 ppm
Aug 10 22:41:46 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401706, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:46.176206 3821 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:41:46.177176 3821 webserver.cc:460] Webserver started at http://127.19.202.67:41339/ using document root <none> and password file <none>
I20260810 22:41:46.177677 3821 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:46.177747 3821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:46.181357 3821 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:41:46.183363 3836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:46.184543 3821 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:41:46.184636 3821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "04a18f23d5364dee970a4b2d2446bd74"
format_stamp: "Formatted at 2026-08-10 22:41:45 on dist-test-slave-1x4g"
server_key: "b73021a7a18e0c7b864e639c5aa19b5b"
server_key_iv: "098ea57f80886e02857a1b0bd5c2fa19"
server_key_version: "encryptionkey@0"
I20260810 22:41:46.185223 3821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:46.198006 3821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:46.200963 3821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:46.201097 3821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:46.201560 3821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:46.202252 3821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:46.202296 3821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:46.202329 3821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:46.202349 3821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:46.211428 3821 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:39121
I20260810 22:41:46.211467 3957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:39121 every 8 connection(s)
I20260810 22:41:46.212424 3821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:41:46.213205 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 3821
I20260810 22:41:46.222165 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.213952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36040 (local address 127.19.202.126:39375)
0810 22:41:46.214098 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.214104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.214549 (+ 445us) server_negotiation.cc:407] Connection header received
0810 22:41:46.214605 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.214608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.214656 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.214746 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:46.215497 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.216039 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.217003 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.217231 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.217343 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.218754 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:46.218757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:46.218759 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:46.218781 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:46.220317 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:46.220771 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:46.220774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:46.220775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:46.220901 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:46.221216 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:46.221219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:46.221220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:46.221446 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:46.221659 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.221860 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.221984 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
Aug 10 22:41:46 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401706, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:41:46.227165 3961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.214396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:59403 (local address 127.19.202.126:39375)
0810 22:41:46.214634 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.214637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.216860 (+ 2223us) server_negotiation.cc:407] Connection header received
0810 22:41:46.217009 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.217012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.217061 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.217148 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:46.218393 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.218945 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.220683 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.221019 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.221127 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.223681 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:46.223685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:46.223687 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:46.223717 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:46.225340 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:46.225828 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:46.225831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:46.225833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:46.225923 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:46.226225 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:46.226227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:46.226228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:46.226399 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:46.226472 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.226968 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.227058 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
I20260810 22:41:46.227909 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.214891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:59403)
0810 22:41:46.215366 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.215426 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.216748 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.217315 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.217321 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.217709 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:46.218253 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.218267 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.219103 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.219106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.220562 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.220572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.221184 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.221190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.221396 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.221952 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:46.221971 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:46.223528 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:46.225462 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:46.225467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:46.225477 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:46.225729 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:46.226010 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:46.226012 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:46.226013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:46.226142 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:46.226493 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:46.226497 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:46.226702 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.226860 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.227081 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":137,"threads_started":1}
I20260810 22:41:46.229099 3958 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39375
I20260810 22:41:46.229359 3958 heartbeater.cc:464] Registering TS with master...
I20260810 22:41:46.229827 3958 heartbeater.cc:511] Master 127.19.202.126:39375 requested a full tablet report, sending...
I20260810 22:41:46.230785 3466 ts_manager.cc:194] Registered new tserver with Master: 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121)
I20260810 22:41:46.231308 3466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:59403
I20260810 22:41:46.236311 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:41:46.239951 3961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.236977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36048 (local address 127.19.202.126:39375)
0810 22:41:46.237078 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.237081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.237272 (+ 191us) server_negotiation.cc:407] Connection header received
0810 22:41:46.237374 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.237376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.237457 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.237539 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:41:46.237706 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.238178 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.238891 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.239079 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.239153 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.239388 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:41:46.239679 (+ 291us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:41:46.239757 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.239815 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.239857 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:41:46.240553 3466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:41:46.243837 3466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:41:46.251829 3600 tablet_service.cc:1467] Processing CreateTablet for tablet f1bd1374ce6b4d8fb677b8631dae374e (DEFAULT_TABLE table=test-table [id=719b1c05305f4a749734714d599cd042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:46.252480 3600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1bd1374ce6b4d8fb677b8631dae374e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:46.255223 3553 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Bootstrap starting.
I20260810 22:41:46.255280 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.250217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39121 (local address 127.0.0.1:35178)
0810 22:41:46.250753 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.250767 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.250855 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.251304 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.251307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.251325 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:46.251414 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.251420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.252600 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.252604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.253758 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.253771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.254837 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.254840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.254928 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.254968 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.255065 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.255143 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":53}
I20260810 22:41:46.255348 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.250401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35178 (local address 127.19.202.67:39121)
0810 22:41:46.250724 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.250734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.250851 (+ 117us) server_negotiation.cc:407] Connection header received
0810 22:41:46.250959 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.250965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.251172 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.251326 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:46.251511 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.252459 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.253948 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.254747 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.254838 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.254983 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.255095 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.255223 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":141,"threads_started":1}
I20260810 22:41:46.256100 3553 tablet_bootstrap.cc:654] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:46.256354 3968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.250515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39033 (local address 127.0.0.1:56170)
0810 22:41:46.250784 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.250796 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.250862 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.251728 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.251731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.251750 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:46.251829 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.251834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.253239 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.253242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.254618 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.254629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.255907 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.255911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.256016 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.256054 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.256135 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.256209 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":54,"spinlock_wait_cycles":1792,"thread_start_us":95,"threads_started":1}
I20260810 22:41:46.256924 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.250740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.19.202.66:39033)
0810 22:41:46.251274 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.251286 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.251308 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:41:46.251388 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.251393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.251575 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.251727 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:46.251938 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.253095 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.254761 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.255608 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.255713 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.255840 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.256561 (+ 721us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.256723 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":173,"threads_started":1}
I20260810 22:41:46.257807 3553 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: No bootstrap required, opened a new log
I20260810 22:41:46.257915 3553 ts_tablet_manager.cc:1397] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 22:41:46.258285 3553 raft_consensus.cc:348] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.258416 3553 raft_consensus.cc:374] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:46.258471 3553 raft_consensus.cc:729] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd70afdba0c14e53b4ddff55d795db15, State: Initialized, Role: FOLLOWER
I20260810 22:41:46.258605 3553 consensus_queue.cc:260] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.258947 3553 ts_tablet_manager.cc:1428] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 22:41:46.259913 3890 tablet_service.cc:1467] Processing CreateTablet for tablet f1bd1374ce6b4d8fb677b8631dae374e (DEFAULT_TABLE table=test-table [id=719b1c05305f4a749734714d599cd042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:46.261029 3890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1bd1374ce6b4d8fb677b8631dae374e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:46.262984 3749 tablet_service.cc:1467] Processing CreateTablet for tablet f1bd1374ce6b4d8fb677b8631dae374e (DEFAULT_TABLE table=test-table [id=719b1c05305f4a749734714d599cd042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:41:46.263978 3749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1bd1374ce6b4d8fb677b8631dae374e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:41:46.267671 3971 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Bootstrap starting.
I20260810 22:41:46.270411 3972 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Bootstrap starting.
I20260810 22:41:46.270442 3971 tablet_bootstrap.cc:654] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:46.271221 3971 log.cc:824] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:46.272318 3972 tablet_bootstrap.cc:654] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Neither blocks nor log segments found. Creating new log.
I20260810 22:41:46.272993 3971 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: No bootstrap required, opened a new log
I20260810 22:41:46.273098 3972 log.cc:824] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:46.273197 3971 ts_tablet_manager.cc:1397] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 22:41:46.274869 3972 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: No bootstrap required, opened a new log
I20260810 22:41:46.275094 3972 ts_tablet_manager.cc:1397] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 22:41:46.276196 3971 raft_consensus.cc:348] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.276403 3971 raft_consensus.cc:374] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:46.276443 3971 raft_consensus.cc:729] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a18f23d5364dee970a4b2d2446bd74, State: Initialized, Role: FOLLOWER
I20260810 22:41:46.276938 3971 consensus_queue.cc:260] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.277563 3958 heartbeater.cc:503] Master 127.19.202.126:39375 was elected leader, sending a full tablet report...
I20260810 22:41:46.277873 3971 ts_tablet_manager.cc:1428] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:41:46.278252 3972 raft_consensus.cc:348] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.278497 3972 raft_consensus.cc:374] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:41:46.278565 3972 raft_consensus.cc:729] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb43aea1d0c42f4a5655b52a9509ce6, State: Initialized, Role: FOLLOWER
I20260810 22:41:46.278972 3972 consensus_queue.cc:260] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.279580 3818 heartbeater.cc:503] Master 127.19.202.126:39375 was elected leader, sending a full tablet report...
I20260810 22:41:46.279831 3972 ts_tablet_manager.cc:1428] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260810 22:41:46.283713 3819 tablet.cc:2369] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:41:46.332293 3669 tablet.cc:2369] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:41:46.463470 3959 tablet.cc:2369] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:41:46.465714 3975 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:46.465929 3975 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.467130 3975 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033), dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525)
I20260810 22:41:46.471208 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.467396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39033 (local address 127.19.202.67:47839)
0810 22:41:46.467527 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.467542 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.467637 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.467923 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.467930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.467947 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:46.468131 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.468138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.469056 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.469060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.469961 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.469971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.470719 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.470723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.470830 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.470857 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.470940 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.471006 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"spinlock_wait_cycles":19456}
I20260810 22:41:46.471222 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.467500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:47839 (local address 127.19.202.66:39033)
0810 22:41:46.467636 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.467641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.467663 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:41:46.467727 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.467730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.467789 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.467881 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:46.468263 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.468945 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.470093 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.470612 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.470701 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.470743 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.470976 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.471033 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:41:46.471616 3978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.467401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41525 (local address 127.19.202.67:47841)
0810 22:41:46.467904 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.467917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.468039 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.468358 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.468361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.468379 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:46.468501 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.468508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.469383 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.469385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.470242 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.470252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.471167 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.471171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.471288 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.471313 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.471404 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.471484 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":116,"spinlock_wait_cycles":768,"thread_start_us":118,"threads_started":1}
I20260810 22:41:46.471652 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.467590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:47841 (local address 127.19.202.65:41525)
0810 22:41:46.467880 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.467886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.467994 (+ 108us) server_negotiation.cc:407] Connection header received
0810 22:41:46.468141 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.468144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.468222 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.468326 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:46.468613 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.469268 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.470372 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.471012 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.471136 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.471324 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.471453 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.471530 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":94,"threads_started":1}
I20260810 22:41:46.471935 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bd1374ce6b4d8fb677b8631dae374e" candidate_uuid: "04a18f23d5364dee970a4b2d2446bd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" is_pre_election: true
I20260810 22:41:46.472138 3977 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:46.472275 3620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bd1374ce6b4d8fb677b8631dae374e" candidate_uuid: "04a18f23d5364dee970a4b2d2446bd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd70afdba0c14e53b4ddff55d795db15" is_pre_election: true
I20260810 22:41:46.472312 3977 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.472568 3620 raft_consensus.cc:2454] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a18f23d5364dee970a4b2d2446bd74 in term 0.
I20260810 22:41:46.472738 3769 raft_consensus.cc:2454] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a18f23d5364dee970a4b2d2446bd74 in term 0.
I20260810 22:41:46.473127 3839 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a18f23d5364dee970a4b2d2446bd74, dd70afdba0c14e53b4ddff55d795db15; no voters:
I20260810 22:41:46.473417 3975 raft_consensus.cc:2781] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:41:46.473503 3975 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:41:46.473563 3975 raft_consensus.cc:3037] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:46.473598 3977 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121), dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525)
I20260810 22:41:46.474987 3975 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.475330 3975 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [CANDIDATE]: Term 1 election: Requested vote from peers ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033), dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525)
I20260810 22:41:46.475659 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bd1374ce6b4d8fb677b8631dae374e" candidate_uuid: "04a18f23d5364dee970a4b2d2446bd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6"
I20260810 22:41:46.475781 3769 raft_consensus.cc:3037] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:46.475936 3620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bd1374ce6b4d8fb677b8631dae374e" candidate_uuid: "04a18f23d5364dee970a4b2d2446bd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd70afdba0c14e53b4ddff55d795db15"
I20260810 22:41:46.476058 3620 raft_consensus.cc:3037] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:41:46.477181 3769 raft_consensus.cc:2454] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a18f23d5364dee970a4b2d2446bd74 in term 1.
I20260810 22:41:46.477413 3620 raft_consensus.cc:2454] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a18f23d5364dee970a4b2d2446bd74 in term 1.
I20260810 22:41:46.477512 3838 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a18f23d5364dee970a4b2d2446bd74, ccb43aea1d0c42f4a5655b52a9509ce6; no voters:
I20260810 22:41:46.477691 3975 raft_consensus.cc:2781] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:41:46.477923 3975 raft_consensus.cc:686] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 1 LEADER]: Becoming Leader. State: Replica: 04a18f23d5364dee970a4b2d2446bd74, State: Running, Role: LEADER
I20260810 22:41:46.478091 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.473746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39121 (local address 127.19.202.66:48091)
0810 22:41:46.473935 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.473949 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.474026 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.474374 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.474377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.474391 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:46.474513 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.474519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.475419 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.475424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.476641 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.476653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.477624 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.477631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.477751 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.477776 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.477869 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.477975 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1}
I20260810 22:41:46.478104 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.473808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:48091 (local address 127.19.202.67:39121)
0810 22:41:46.474168 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.474172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.474183 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:41:46.474220 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.474222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.474257 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.474329 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:46.474629 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.475269 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.476811 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.477404 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.477511 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.477562 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.477899 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.477997 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260810 22:41:46.478312 3975 consensus_queue.cc:237] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:46.478643 3910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bd1374ce6b4d8fb677b8631dae374e" candidate_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a18f23d5364dee970a4b2d2446bd74" is_pre_election: true
I20260810 22:41:46.478986 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.473873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41525 (local address 127.19.202.66:46629)
0810 22:41:46.474261 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.474272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:46.474385 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:46.474717 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:46.474719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:46.474732 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:41:46.474817 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.474820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.475770 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.475773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.477012 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:46.477026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.478579 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.478587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:46.478660 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.478676 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 22:41:46.478771 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 22:41:46.478848 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":121,"thread_start_us":136,"threads_started":1}
I20260810 22:41:46.479224 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.473989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:46629 (local address 127.19.202.65:41525)
0810 22:41:46.474101 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.474105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.474464 (+ 359us) server_negotiation.cc:407] Connection header received
0810 22:41:46.474510 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.474512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.474560 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.474626 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:41:46.474945 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.475643 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.477158 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.478076 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.478444 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.478501 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.478834 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.478956 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:41:46.479537 3620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bd1374ce6b4d8fb677b8631dae374e" candidate_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd70afdba0c14e53b4ddff55d795db15" is_pre_election: true
I20260810 22:41:46.479718 3620 raft_consensus.cc:2379] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccb43aea1d0c42f4a5655b52a9509ce6 in current term 1: Already voted for candidate 04a18f23d5364dee970a4b2d2446bd74 in this term.
I20260810 22:41:46.480139 3698 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccb43aea1d0c42f4a5655b52a9509ce6; no voters: 04a18f23d5364dee970a4b2d2446bd74, dd70afdba0c14e53b4ddff55d795db15
I20260810 22:41:46.480592 3977 raft_consensus.cc:2726] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 22:41:46.481859 3466 catalog_manager.cc:5495] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "04a18f23d5364dee970a4b2d2446bd74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } health_report { overall_health: UNKNOWN } } }
I20260810 22:41:46.549431 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.546178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35184 (local address 127.19.202.67:39121)
0810 22:41:46.546314 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:46.546318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:46.546348 (+ 30us) server_negotiation.cc:407] Connection header received
0810 22:41:46.546433 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:46.546436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:46.546490 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:46.546576 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:41:46.546788 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.547269 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.548044 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:46.548258 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:46.548334 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:46.548602 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:41:46.549015 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:41:46.549103 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:46.549169 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 22:41:46.549219 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 22:41:46.555250 3769 raft_consensus.cc:1261] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Refusing update from remote peer 04a18f23d5364dee970a4b2d2446bd74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:46.555292 3620 raft_consensus.cc:1261] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Refusing update from remote peer 04a18f23d5364dee970a4b2d2446bd74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:41:46.555953 3981 consensus_queue.cc:1035] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:41:46.556172 3975 consensus_queue.cc:1035] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:41:46.569060 3986 mvcc.cc:205] Tried to move back new op lower bound from 7317101390043267072 to 7317101389738905600. Current Snapshot: MvccSnapshot[applied={T|T < 7317101390043267072 or (T in {7317101390043267072})}]
I20260810 22:41:46.570783 3464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:41:46.570924 3464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:41:46.573292 3464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Sending DeleteTablet for 3 replicas of tablet f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:46.573974 3890 tablet_service.cc:1514] Processing DeleteTablet for tablet f1bd1374ce6b4d8fb677b8631dae374e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:46 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:35178
I20260810 22:41:46.573997 3749 tablet_service.cc:1514] Processing DeleteTablet for tablet f1bd1374ce6b4d8fb677b8631dae374e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:46 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:56170
I20260810 22:41:46.574067 3600 tablet_service.cc:1514] Processing DeleteTablet for tablet f1bd1374ce6b4d8fb677b8631dae374e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:41:46 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:39660
I20260810 22:41:46.574196 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 3508
I20260810 22:41:46.578238 3996 tablet_replica.cc:330] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: stopping tablet replica
I20260810 22:41:46.578555 3996 raft_consensus.cc:2229] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:41:46.578887 3996 raft_consensus.cc:2258] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:41:46.580170 3996 ts_tablet_manager.cc:1905] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:41:46.582816 3996 ts_tablet_manager.cc:1918] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:41:46.582935 3996 log.cc:1196] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:46.583212 3995 tablet_replica.cc:330] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: stopping tablet replica
I20260810 22:41:46.583308 3996 ts_tablet_manager.cc:1939] T f1bd1374ce6b4d8fb677b8631dae374e P ccb43aea1d0c42f4a5655b52a9509ce6: Deleting consensus metadata
I20260810 22:41:46.583702 3995 raft_consensus.cc:2229] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:41:46.584105 3995 raft_consensus.cc:2258] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:41:46.584563 3448 catalog_manager.cc:4841] TS ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033): tablet f1bd1374ce6b4d8fb677b8631dae374e (table test-table [id=719b1c05305f4a749734714d599cd042]) successfully deleted
I20260810 22:41:46.585521 3995 ts_tablet_manager.cc:1905] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 22:41:46.586884 3449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:41525 (error 108)
I20260810 22:41:46.587399 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 3674
I20260810 22:41:46.588143 3995 ts_tablet_manager.cc:1918] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:41:46.588197 3968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:46.587805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41525 (local address 127.0.0.1:39672)
0810 22:41:46.588034 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:46.588109 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:41525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260810 22:41:46.588236 3995 log.cc:1196] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:46.588626 3995 ts_tablet_manager.cc:1939] T f1bd1374ce6b4d8fb677b8631dae374e P 04a18f23d5364dee970a4b2d2446bd74: Deleting consensus metadata
W20260810 22:41:46.588477 3449 catalog_manager.cc:4584] TS dd70afdba0c14e53b4ddff55d795db15 (127.19.202.65:41525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1bd1374ce6b4d8fb677b8631dae374e: Network error: Client connection negotiation failed: client connection to 127.19.202.65:41525: connect: Connection refused (error 111)
I20260810 22:41:46.589818 3447 catalog_manager.cc:4841] TS 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121): tablet f1bd1374ce6b4d8fb677b8631dae374e (table test-table [id=719b1c05305f4a749734714d599cd042]) successfully deleted
I20260810 22:41:46.598892 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 3821
I20260810 22:41:46.607687 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 3430
I20260810 22:41:46.615655 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39375
--webserver_interface=127.19.202.126
--webserver_port=35565
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:39375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:41:46.724761 3998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:46.725030 3998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:46.725080 3998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:46.728935 3998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:41:46.728982 3998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:46.729001 3998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:46.729017 3998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:41:46.729032 3998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:41:46.734162 3998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:39375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=35565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:46.735146 3998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:46.736115 3998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:46.744686 4004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:46.744699 4003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:46.744931 3998 server_base.cc:1034] running on GCE node
W20260810 22:41:46.744719 4007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:46.745548 3998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:46.746690 3998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:46.747879 3998 hybrid_clock.cc:648] HybridClock initialized: now 1786401706747849 us; error 39 us; skew 500 ppm
Aug 10 22:41:46 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401706, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:46.750705 3998 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:41:46.751634 3998 webserver.cc:460] Webserver started at http://127.19.202.126:35565/ using document root <none> and password file <none>
I20260810 22:41:46.752189 3998 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:46.752285 3998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:46.756114 3998 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:41:46.758268 4015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:46.759421 3998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:41:46.759572 3998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "73774eb8341240a79e8f700077e03ddd"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "4826e79a6d9f7e46647a502f4f8412db"
server_key_iv: "e91e1cb0bc39176bc1ff7947e7866c27"
server_key_version: "encryptionkey@0"
I20260810 22:41:46.760097 3998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:46.803493 3998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:46.807035 3998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:46.807255 3998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:46.814778 3998 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:39375
I20260810 22:41:46.815150 4071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:39375 every 8 connection(s)
I20260810 22:41:46.815961 3998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:41:46.820242 4072 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:41:46.821862 4072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Bootstrap starting.
I20260810 22:41:46.822373 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 3998
I20260810 22:41:46.822793 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:41525
--local_ip_for_outbound_sockets=127.19.202.65
--tserver_master_addrs=127.19.202.126:39375
--webserver_port=45607
--webserver_interface=127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:46.846297 4072 log.cc:824] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:46.860850 4072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:41:46.861264 4072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd: Bootstrap complete.
I20260810 22:41:46.866775 4072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:46.867349 4072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73774eb8341240a79e8f700077e03ddd, State: Initialized, Role: FOLLOWER
I20260810 22:41:46.867918 4072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:46.868081 4072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:41:46.868153 4072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:41:46.868252 4072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 2 FOLLOWER]: Advancing to term 3
I20260810 22:41:46.870648 4072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:46.871033 4072 leader_election.cc:302] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73774eb8341240a79e8f700077e03ddd; no voters:
I20260810 22:41:46.871357 4072 leader_election.cc:288] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 22:41:46.871497 4076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 3 FOLLOWER]: Leader election won for term 3
I20260810 22:41:46.871765 4076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [term 3 LEADER]: Becoming Leader. State: Replica: 73774eb8341240a79e8f700077e03ddd, State: Running, Role: LEADER
I20260810 22:41:46.872093 4076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } }
I20260810 22:41:46.872604 4072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:41:46.873019 4078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73774eb8341240a79e8f700077e03ddd. Latest consensus state: current_term: 3 leader_uuid: "73774eb8341240a79e8f700077e03ddd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } } }
I20260810 22:41:46.873157 4078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:46.873425 4077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "73774eb8341240a79e8f700077e03ddd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73774eb8341240a79e8f700077e03ddd" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39375 } } }
I20260810 22:41:46.873504 4077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73774eb8341240a79e8f700077e03ddd [sys.catalog]: This master's current role is: LEADER
I20260810 22:41:46.873582 4080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:41:46.878719 4080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:41:46.879751 4080 catalog_manager.cc:1259] Loaded cluster ID: dd95637a32284e74b7d6bc2cf8984257
I20260810 22:41:46.880174 4080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 22:41:46.881253 4080 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f500d8d69ad google::LogMessage::Fail() at ??:0
@ 0x7f500d8d988a google::LogMessage::SendToLog() at ??:0
@ 0x7f500d8d64de google::LogMessage::Flush() at ??:0
@ 0x7f500d8d7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5018215fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5018216d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5018213579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5018257640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f500fb283d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f500fb28c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f500fb2a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f500fb19512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f501115d6db start_thread at ??:0
@ 0x7f500c3d461f clone at ??:0
W20260810 22:41:46.979476 4074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:46.979700 4074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:46.979740 4074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:46.983747 4074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:46.983800 4074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:46.983891 4074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:41:46.989239 4074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:41525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=45607
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:46.990239 4074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:46.991261 4074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:47.001031 4115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:47.001049 4114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:47.001051 4117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:47.002833 4074 server_base.cc:1034] running on GCE node
I20260810 22:41:47.003314 4074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:47.004112 4074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:47.005321 4074 hybrid_clock.cc:648] HybridClock initialized: now 1786401707005292 us; error 36 us; skew 500 ppm
Aug 10 22:41:47 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401707, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:47.008911 4074 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:41:47.010017 4074 webserver.cc:460] Webserver started at http://127.19.202.65:45607/ using document root <none> and password file <none>
I20260810 22:41:47.010707 4074 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:47.010810 4074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:47.015803 4074 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:41:47.018563 4126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:47.020064 4074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:41:47.020188 4074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "dd70afdba0c14e53b4ddff55d795db15"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "1d331bfc27f06bc43ee6d425ac48acb2"
server_key_iv: "7f9f249bf10dbd71d81bfb3f37040456"
server_key_version: "encryptionkey@0"
I20260810 22:41:47.020946 4074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:47.035096 4074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:47.038738 4074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:47.038954 4074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:47.039654 4074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:47.041071 4133 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 22:41:47.045331 4074 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 22:41:47.045410 4074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260810 22:41:47.045480 4074 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 22:41:47.047771 4074 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 22:41:47.047835 4074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 22:41:47.048171 4133 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Bootstrap starting.
I20260810 22:41:47.059657 4074 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:41525
I20260810 22:41:47.060843 4074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:41:47.061457 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 4074
I20260810 22:41:47.061862 4251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:41525 every 8 connection(s)
I20260810 22:41:47.062043 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:39033
--local_ip_for_outbound_sockets=127.19.202.66
--tserver_master_addrs=127.19.202.126:39375
--webserver_port=36329
--webserver_interface=127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:47.071059 4133 log.cc:824] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Log is configured to *not* fsync() on all Append() calls
I20260810 22:41:47.080060 4133 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:41:47.080559 4133 tablet_bootstrap.cc:492] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Bootstrap complete.
I20260810 22:41:47.082334 4133 ts_tablet_manager.cc:1397] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.008s
I20260810 22:41:47.085851 4133 raft_consensus.cc:348] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:47.086452 4133 raft_consensus.cc:729] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd70afdba0c14e53b4ddff55d795db15, State: Initialized, Role: FOLLOWER
I20260810 22:41:47.086997 4133 consensus_queue.cc:260] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:47.088059 4133 ts_tablet_manager.cc:1428] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260810 22:41:47.129477 4255 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:41:47.064222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:44429)
0810 22:41:47.065186 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.065262 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:47.066455 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:47.128626 (+ 62171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":173,"threads_started":1}
W20260810 22:41:47.130018 4252 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:39375 (0 consecutive failures): Network error: Failed to ping master at 127.19.202.126:39375: Client connection negotiation failed: client connection to 127.19.202.126:39375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 22:41:47.130870 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.130614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:41127)
0810 22:41:47.130755 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.130818 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 22:41:47.131513 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.131310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:34177)
0810 22:41:47.131417 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.131469 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 22:41:47.131683 4252 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:41:47.184057 4254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:47.184332 4254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:47.184407 4254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:47.188433 4254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:47.188517 4254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:47.188603 4254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:41:47.194031 4254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:39033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=36329
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:47.195044 4254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:47.196107 4254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:47.205801 4263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:47.205808 4266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:47.205965 4264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:47.206123 4254 server_base.cc:1034] running on GCE node
I20260810 22:41:47.206604 4254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:47.207248 4254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:47.208424 4254 hybrid_clock.cc:648] HybridClock initialized: now 1786401707208393 us; error 43 us; skew 500 ppm
Aug 10 22:41:47 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401707, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:47.211231 4254 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:41:47.212093 4254 webserver.cc:460] Webserver started at http://127.19.202.66:36329/ using document root <none> and password file <none>
I20260810 22:41:47.212663 4254 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:47.212774 4254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:47.216468 4254 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 22:41:47.218526 4273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:47.219688 4254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 22:41:47.219830 4254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "ccb43aea1d0c42f4a5655b52a9509ce6"
format_stamp: "Formatted at 2026-08-10 22:41:44 on dist-test-slave-1x4g"
server_key: "6a6b537490a63eee23f71a5937e92826"
server_key_iv: "1f2081fc4894ed5e89dbb833b47137cc"
server_key_version: "encryptionkey@0"
I20260810 22:41:47.220427 4254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:47.235126 4254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:47.237807 4254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:47.238013 4254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:47.238605 4254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:47.239387 4254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:47.239461 4254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:47.239530 4254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:47.239575 4254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:47.248338 4254 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:39033
I20260810 22:41:47.248409 4396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:39033 every 8 connection(s)
I20260810 22:41:47.249368 4254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:41:47.252074 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 4254
I20260810 22:41:47.252497 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:39121
--local_ip_for_outbound_sockets=127.19.202.67
--tserver_master_addrs=127.19.202.126:39375
--webserver_port=41339
--webserver_interface=127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:41:47.253334 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.251307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:55645)
0810 22:41:47.251752 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.251874 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
W20260810 22:41:47.253791 4397 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:39375 (0 consecutive failures): Network error: Failed to ping master at 127.19.202.126:39375: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
I20260810 22:41:47.254506 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.254197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:47679)
0810 22:41:47.254350 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.254426 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 22:41:47.255201 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.254994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:41753)
0810 22:41:47.255106 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.255150 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 22:41:47.255463 4397 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:41:47.363984 4400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:41:47.364195 4400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:41:47.364234 4400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:41:47.368140 4400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:41:47.368189 4400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:41:47.368283 4400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:41:47.373431 4400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:39121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=41339
--tserver_master_addrs=127.19.202.126:39375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:41:47.374325 4400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:41:47.375262 4400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:41:47.384457 4408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:41:47.384485 4406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:47.384696 4400 server_base.cc:1034] running on GCE node
W20260810 22:41:47.384464 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:41:47.385275 4400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:41:47.385869 4400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:41:47.387043 4400 hybrid_clock.cc:648] HybridClock initialized: now 1786401707387016 us; error 38 us; skew 500 ppm
Aug 10 22:41:47 dist-test-slave-1x4g krb5kdc[2833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401707, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:41:47.389892 4400 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:41:47.390738 4400 webserver.cc:460] Webserver started at http://127.19.202.67:41339/ using document root <none> and password file <none>
I20260810 22:41:47.391299 4400 fs_manager.cc:359] Metadata directory not provided
I20260810 22:41:47.391389 4400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:41:47.394965 4400 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:41:47.396967 4417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:47.398116 4400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:41:47.398252 4400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "04a18f23d5364dee970a4b2d2446bd74"
format_stamp: "Formatted at 2026-08-10 22:41:45 on dist-test-slave-1x4g"
server_key: "b73021a7a18e0c7b864e639c5aa19b5b"
server_key_iv: "098ea57f80886e02857a1b0bd5c2fa19"
server_key_version: "encryptionkey@0"
I20260810 22:41:47.398768 4400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:41:47.435700 4400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:41:47.438740 4400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:41:47.438946 4400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:41:47.439545 4400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:41:47.440397 4400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:41:47.440471 4400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:47.440552 4400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:41:47.440588 4400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:41:47.449961 4400 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:39121
I20260810 22:41:47.450031 4542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:39121 every 8 connection(s)
I20260810 22:41:47.451025 4400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:41:47.454838 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.453152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:33499)
0810 22:41:47.453584 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.453692 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
W20260810 22:41:47.455339 4545 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:39375 (0 consecutive failures): Network error: Failed to ping master at 127.19.202.126:39375: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
I20260810 22:41:47.455874 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.455681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:51489)
0810 22:41:47.455781 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.455824 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 22:41:47.456442 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:47.456259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:57551)
0810 22:41:47.456350 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:47.456389 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 22:41:47.456670 4545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 22:41:47.460520 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 4400
I20260810 22:41:48.132705 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:48.132267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:36795)
0810 22:41:48.132552 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:48.132605 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":88,"threads_started":1}
I20260810 22:41:48.256498 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:48.256011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:51845)
0810 22:41:48.256319 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:48.256385 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":115,"threads_started":1}
I20260810 22:41:48.457556 4551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:48.457180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:36543)
0810 22:41:48.457406 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:48.457457 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260810 22:41:48.716367 4552 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:48.716578 4552 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:48.717725 4552 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121), ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033)
Aug 10 22:41:48 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401707, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.66@KRBTEST.COM
Aug 10 22:41:48 dist-test-slave-1x4g krb5kdc[2833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401707, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.67@KRBTEST.COM
I20260810 22:41:48.743068 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:48.717955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39033 (local address 127.19.202.65:44135)
0810 22:41:48.718379 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:48.718394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:48.718498 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:48.720458 (+ 1960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:48.720464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:48.721154 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:48.722045 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:48.722069 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.723987 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.723990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:48.725841 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:48.725855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.726860 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.726865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:48.727207 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:48.736436 (+ 9229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:48.736462 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:48.738266 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:48.740469 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:48.740476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:48.740491 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:48.740842 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:48.741204 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:48.741208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:48.741210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:48.741357 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:48.742013 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:48.742020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:48.742256 (+ 236us) client_negotiation.cc:769] Sending connection context
0810 22:41:48.742463 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 22:41:48.742862 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":517,"spinlock_wait_cycles":10112,"thread_start_us":73,"threads_started":1}
I20260810 22:41:48.743083 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:48.718216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:44135 (local address 127.19.202.66:39033)
0810 22:41:48.718613 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:48.718632 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:48.718664 (+ 32us) server_negotiation.cc:407] Connection header received
0810 22:41:48.719533 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:48.719544 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:48.719989 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:48.720351 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:48.722241 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.723803 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.725985 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.726713 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.726873 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:48.738540 (+ 11667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:48.738546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:48.738560 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:48.738590 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:48.740335 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:48.740963 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:48.740968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:48.740972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:48.741092 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:48.741463 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:48.741471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:48.741472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:48.741893 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:48.742040 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:48.742632 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 22:41:48.742922 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":96,"threads_started":1}
I20260810 22:41:48.743127 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:48.718081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:40967 (local address 127.19.202.67:39121)
0810 22:41:48.718548 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0810 22:41:48.718574 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 22:41:48.718619 (+ 45us) server_negotiation.cc:407] Connection header received
0810 22:41:48.719538 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:41:48.719544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:41:48.719989 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:41:48.720351 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:41:48.722241 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.723803 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.725985 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.726713 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.726873 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:48.738841 (+ 11968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:41:48.738846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:41:48.738858 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:41:48.738916 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:41:48.740732 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:48.741246 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:48.741252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:48.741257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:48.741378 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:41:48.741677 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:41:48.741680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:41:48.741682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:41:48.742057 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:41:48.742198 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 22:41:48.742640 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 22:41:48.742933 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":104,"threads_started":1}
I20260810 22:41:48.743116 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:48.717901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39121 (local address 127.19.202.65:40967)
0810 22:41:48.718247 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:48.718261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:41:48.718365 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:41:48.720458 (+ 2093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:41:48.720464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:41:48.721158 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:41:48.722045 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:48.722069 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.723984 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.723988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:48.725841 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:41:48.725855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:41:48.726860 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:41:48.726865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:41:48.727207 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:41:48.736441 (+ 9234us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:41:48.736458 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:41:48.738634 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:41:48.740914 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:48.740917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:48.740919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:48.741120 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:48.741484 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:41:48.741488 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:41:48.741490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:41:48.741579 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:41:48.742169 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:41:48.742172 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:41:48.742281 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 22:41:48.742460 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 22:41:48.742957 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":566,"spinlock_wait_cycles":16768,"thread_start_us":96,"threads_started":1}
W20260810 22:41:48.746016 4127 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
W20260810 22:41:48.746176 4128 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:48.746249 4128 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd70afdba0c14e53b4ddff55d795db15; no voters: 04a18f23d5364dee970a4b2d2446bd74, ccb43aea1d0c42f4a5655b52a9509ce6
I20260810 22:41:48.746558 4552 raft_consensus.cc:2726] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:41:49.133757 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:49.133444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:55719)
0810 22:41:49.133605 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:49.133665 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 22:41:49.257591 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:49.257213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:59539)
0810 22:41:49.257440 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:49.257500 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260810 22:41:49.458676 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:49.458258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:51379)
0810 22:41:49.458533 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:49.458589 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":75,"threads_started":1}
I20260810 22:41:50.134980 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:50.134547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:59035)
0810 22:41:50.134798 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:50.134876 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260810 22:41:50.258606 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:50.258238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:54945)
0810 22:41:50.258475 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:50.258525 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":68,"threads_started":1}
I20260810 22:41:50.296077 4563 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:50.296207 4563 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:50.296701 4563 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121), ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033)
W20260810 22:41:50.297567 4128 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
W20260810 22:41:50.297696 4127 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:50.297791 4127 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd70afdba0c14e53b4ddff55d795db15; no voters: 04a18f23d5364dee970a4b2d2446bd74, ccb43aea1d0c42f4a5655b52a9509ce6
I20260810 22:41:50.297963 4563 raft_consensus.cc:2726] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:41:50.459739 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:50.459372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:40503)
0810 22:41:50.459613 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:50.459663 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
I20260810 22:41:51.136142 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:51.135693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:54855)
0810 22:41:51.135959 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:51.136042 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
I20260810 22:41:51.259688 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:51.259291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:33735)
0810 22:41:51.259558 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:51.259609 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":70,"threads_started":1}
I20260810 22:41:51.460889 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:51.460450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:48389)
0810 22:41:51.460702 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:51.460755 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260810 22:41:52.064061 4570 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:52.064213 4570 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:52.064638 4570 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121), ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033)
W20260810 22:41:52.065487 4127 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
W20260810 22:41:52.065626 4128 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:52.065717 4128 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd70afdba0c14e53b4ddff55d795db15; no voters: 04a18f23d5364dee970a4b2d2446bd74, ccb43aea1d0c42f4a5655b52a9509ce6
I20260810 22:41:52.065882 4570 raft_consensus.cc:2726] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:41:52.137749 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:52.137280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:47001)
0810 22:41:52.137592 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:52.137649 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":96,"threads_started":1}
I20260810 22:41:52.260891 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:52.260438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:58303)
0810 22:41:52.260704 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:52.260754 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
I20260810 22:41:52.461983 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:52.461597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:36069)
0810 22:41:52.461838 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:52.461892 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
I20260810 22:41:53.138943 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:53.138513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:48725)
0810 22:41:53.138778 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:53.138828 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":96,"threads_started":1}
I20260810 22:41:53.261884 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:53.261512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:42189)
0810 22:41:53.261748 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:53.261800 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":69,"threads_started":1}
I20260810 22:41:53.463038 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:53.462680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:60435)
0810 22:41:53.462902 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:53.462953 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260810 22:41:54.140065 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:54.139661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:42065)
0810 22:41:54.139915 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:54.139968 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260810 22:41:54.262981 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:54.262561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:56277)
0810 22:41:54.262837 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:54.262889 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":69,"threads_started":1}
I20260810 22:41:54.464372 4579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:54.463846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:43697)
0810 22:41:54.464210 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:54.464268 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":135,"threads_started":1}
I20260810 22:41:55.141253 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:55.140865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:42167)
0810 22:41:55.141109 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:55.141160 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":71,"threads_started":1}
I20260810 22:41:55.264117 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:55.263656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:45321)
0810 22:41:55.263963 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:55.264022 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":94,"threads_started":1}
I20260810 22:41:55.465735 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:55.465319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:34711)
0810 22:41:55.465584 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:55.465640 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260810 22:41:55.566063 4583 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:55.566253 4583 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:55.566926 4583 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121), ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033)
W20260810 22:41:55.567674 4128 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
W20260810 22:41:55.567787 4127 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:55.567930 4127 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd70afdba0c14e53b4ddff55d795db15; no voters: 04a18f23d5364dee970a4b2d2446bd74, ccb43aea1d0c42f4a5655b52a9509ce6
I20260810 22:41:55.568128 4583 raft_consensus.cc:2726] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:41:56.142360 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:56.141978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:50145)
0810 22:41:56.142214 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:56.142264 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260810 22:41:56.265393 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:56.264981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:34285)
0810 22:41:56.265242 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:56.265298 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260810 22:41:56.466946 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:56.466493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:34299)
0810 22:41:56.466779 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:56.466848 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":90,"threads_started":1}
I20260810 22:41:57.143612 4587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:57.143224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:58275)
0810 22:41:57.143465 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:57.143516 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260810 22:41:57.266536 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:57.266129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:43983)
0810 22:41:57.266379 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:57.266441 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":88,"threads_started":1}
I20260810 22:41:57.468086 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:57.467672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:56427)
0810 22:41:57.467936 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:57.467991 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260810 22:41:58.144770 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:58.144349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:37319)
0810 22:41:58.144598 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:58.144650 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":89,"threads_started":1}
I20260810 22:41:58.267674 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:58.267286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:46327)
0810 22:41:58.267518 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:58.267570 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
I20260810 22:41:58.469205 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:58.468808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:50939)
0810 22:41:58.469070 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:58.469120 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
I20260810 22:41:59.145936 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:59.145508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:48343)
0810 22:41:59.145775 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:59.145831 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260810 22:41:59.268899 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:59.268429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:38017)
0810 22:41:59.268670 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:59.268722 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260810 22:41:59.302258 4597 raft_consensus.cc:482] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:41:59.302438 4597 raft_consensus.cc:504] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04a18f23d5364dee970a4b2d2446bd74" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39121 } } peers { permanent_uuid: "ccb43aea1d0c42f4a5655b52a9509ce6" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39033 } } peers { permanent_uuid: "dd70afdba0c14e53b4ddff55d795db15" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41525 } }
I20260810 22:41:59.302860 4597 leader_election.cc:288] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121), ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033)
W20260810 22:41:59.303618 4128 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ccb43aea1d0c42f4a5655b52a9509ce6 (127.19.202.66:39033): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
W20260810 22:41:59.303699 4127 leader_election.cc:341] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04a18f23d5364dee970a4b2d2446bd74 (127.19.202.67:39121): Not found: Tablet not found: f1bd1374ce6b4d8fb677b8631dae374e
I20260810 22:41:59.303799 4127 leader_election.cc:302] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd70afdba0c14e53b4ddff55d795db15; no voters: 04a18f23d5364dee970a4b2d2446bd74, ccb43aea1d0c42f4a5655b52a9509ce6
I20260810 22:41:59.303949 4597 raft_consensus.cc:2726] T f1bd1374ce6b4d8fb677b8631dae374e P dd70afdba0c14e53b4ddff55d795db15 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:41:59.470244 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:41:59.469846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:39001)
0810 22:41:59.470103 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 22:41:59.470156 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
I20260810 22:42:00.147353 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:00.146795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:33837)
0810 22:42:00.147162 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:00.147228 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":118,"threads_started":1}
I20260810 22:42:00.270097 4600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:00.269639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:53779)
0810 22:42:00.269934 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:00.269993 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":80,"threads_started":1}
I20260810 22:42:00.471335 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:00.470952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:42769)
0810 22:42:00.471196 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:00.471246 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260810 22:42:01.148654 4602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:01.148235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:52319)
0810 22:42:01.148503 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:01.148557 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":98,"threads_started":1}
I20260810 22:42:01.271399 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:01.270938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:46463)
0810 22:42:01.271238 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:01.271298 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
I20260810 22:42:01.472630 4605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:01.472204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:41789)
0810 22:42:01.472459 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:01.472534 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
I20260810 22:42:02.150084 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:02.149608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.65:33367)
0810 22:42:02.149918 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:02.149984 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":104,"threads_started":1}
I20260810 22:42:02.272629 4607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:02.272205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.66:38461)
0810 22:42:02.272454 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:02.272517 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":70,"threads_started":1}
I20260810 22:42:02.466076 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 4074
I20260810 22:42:02.474012 4608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:02.473586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39375 (local address 127.19.202.67:49545)
0810 22:42:02.473863 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:02.473920 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":86,"threads_started":1}
I20260810 22:42:02.475147 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 4254
I20260810 22:42:02.484081 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 4400
2026-08-10T22:42:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19480 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:42:02 dist-test-slave-1x4g krb5kdc[4612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:42:02 dist-test-slave-1x4g krb5kdc[4612](info): set up 2 sockets
Aug 10 22:42:02 dist-test-slave-1x4g krb5kdc[4612](info): commencing operation
krb5kdc: starting...
W20260810 22:42:03.564002 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:42:03 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:42:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:42:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:42:03.689819 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37125
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:42:03.804083 4626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:03.804347 4626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:03.804388 4626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:03.808378 4626 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 22:42:03.808435 4626 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 22:42:03.808451 4626 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 22:42:03.808465 4626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:42:03.808509 4626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:03.808549 4626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:03.808578 4626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:42:03.808617 4626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:42:03.813867 4626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:03.814949 4626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:03.815994 4626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:03.824985 4634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:03.824980 4631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:03.824980 4632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:03.825083 4626 server_base.cc:1034] running on GCE node
I20260810 22:42:03.825868 4626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:03.826982 4626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:03.828167 4626 hybrid_clock.cc:648] HybridClock initialized: now 1786401723828154 us; error 30 us; skew 500 ppm
Aug 10 22:42:03 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401723, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:03.830972 4626 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:42:03.831904 4626 webserver.cc:460] Webserver started at http://127.19.202.126:36175/ using document root <none> and password file <none>
I20260810 22:42:03.832463 4626 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:03.832552 4626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:03.832823 4626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:42:03.834851 4626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "e8593e37e2dd419299520249db4b89da"
format_stamp: "Formatted at 2026-08-10 22:42:03 on dist-test-slave-1x4g"
server_key: "6d1c9467486ee970c2899d6d60b02b62"
server_key_iv: "c874adf321bcd5099179811704690dcc"
server_key_version: "encryptionkey@0"
I20260810 22:42:03.835434 4626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "e8593e37e2dd419299520249db4b89da"
format_stamp: "Formatted at 2026-08-10 22:42:03 on dist-test-slave-1x4g"
server_key: "6d1c9467486ee970c2899d6d60b02b62"
server_key_iv: "c874adf321bcd5099179811704690dcc"
server_key_version: "encryptionkey@0"
I20260810 22:42:03.839169 4626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:42:03.841506 4641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:03.842651 4626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:42:03.842787 4626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "e8593e37e2dd419299520249db4b89da"
format_stamp: "Formatted at 2026-08-10 22:42:03 on dist-test-slave-1x4g"
server_key: "6d1c9467486ee970c2899d6d60b02b62"
server_key_iv: "c874adf321bcd5099179811704690dcc"
server_key_version: "encryptionkey@0"
I20260810 22:42:03.842897 4626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:03.858137 4626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:03.860709 4626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:03.860926 4626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:03.867869 4626 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:37125
I20260810 22:42:03.868146 4699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:37125 every 8 connection(s)
I20260810 22:42:03.868830 4626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:42:03.871536 4700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:42:03.876391 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 4626
I20260810 22:42:03.876530 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:42:03.876904 20265 external_mini_cluster.cc:1428] Setting key 4736be4d6244c35ae8a3b7474a9a0148
I20260810 22:42:03.877313 4700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Bootstrap starting.
I20260810 22:42:03.879724 4700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Neither blocks nor log segments found. Creating new log.
I20260810 22:42:03.880673 4700 log.cc:824] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:03.882680 4700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: No bootstrap required, opened a new log
Aug 10 22:42:03 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:42:03.886996 4700 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:03.887233 4700 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:42:03.887346 4700 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8593e37e2dd419299520249db4b89da, State: Initialized, Role: FOLLOWER
I20260810 22:42:03.887804 4700 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:03.887975 4700 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:42:03.888055 4700 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:42:03.888172 4700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:42:03.889065 4700 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:03.889380 4700 leader_election.cc:302] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8593e37e2dd419299520249db4b89da; no voters:
I20260810 22:42:03.889685 4700 leader_election.cc:288] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:42:03.889747 4708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:42:03.889935 4708 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 LEADER]: Becoming Leader. State: Replica: e8593e37e2dd419299520249db4b89da, State: Running, Role: LEADER
I20260810 22:42:03.890300 4708 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:03.891009 4700 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:42:03.891422 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:03.878253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.19.202.126:37125)
0810 22:42:03.878684 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:03.878699 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:03.878734 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:42:03.879361 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:03.879378 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:03.879680 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:03.879997 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:03.880929 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:03.882044 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:03.883166 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:03.883610 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:03.883723 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:03.886476 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:03.886481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:03.886493 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:03.886517 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:03.888468 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:03.888924 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:03.888930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:03.888935 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:03.889053 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:03.889340 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:03.889343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:03.889345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:03.889790 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:03.890028 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:03.890303 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 22:42:03.890610 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":154,"threads_started":1}
I20260810 22:42:03.892532 4710 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8593e37e2dd419299520249db4b89da. Latest consensus state: current_term: 1 leader_uuid: "e8593e37e2dd419299520249db4b89da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } } }
I20260810 22:42:03.892567 4709 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8593e37e2dd419299520249db4b89da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } } }
I20260810 22:42:03.892824 4709 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: This master's current role is: LEADER
I20260810 22:42:03.893100 4710 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: This master's current role is: LEADER
W20260810 22:42:03.896047 4723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:42:03.896131 4723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:42:03.896214 4722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:42:03.897082 4722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:42:03.902442 4722 catalog_manager.cc:1347] Generated new cluster ID: 595527617bda4cc888db28562993bf3c
I20260810 22:42:03.902513 4722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:42:03.915632 4722 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:42:03.916666 4722 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:42:03.924551 4722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Generated new TSK 0
I20260810 22:42:03.925298 4722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:42:03.957957 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:37125
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:42:04.071206 4729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:04.071457 4729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:04.071499 4729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:04.075333 4729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:04.075387 4729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:04.075441 4729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:42:04.080421 4729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:04.081362 4729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:04.082266 4729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:04.091850 4734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:04.091883 4735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:04.091953 4739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:04.091876 4729 server_base.cc:1034] running on GCE node
I20260810 22:42:04.092420 4729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:04.093093 4729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:04.094278 4729 hybrid_clock.cc:648] HybridClock initialized: now 1786401724094251 us; error 39 us; skew 500 ppm
Aug 10 22:42:04 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401724, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:04.097173 4729 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:42:04.098076 4729 webserver.cc:460] Webserver started at http://127.19.202.65:41875/ using document root <none> and password file <none>
I20260810 22:42:04.098641 4729 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:04.098729 4729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:04.098966 4729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:42:04.101120 4729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "662782d9a7cf46558b4f73392916c5e8"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "95263a6f4e3d7697cb27aaca6949262b"
server_key_iv: "910067664ec431c51b6612ae5ba803de"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.101679 4729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "662782d9a7cf46558b4f73392916c5e8"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "95263a6f4e3d7697cb27aaca6949262b"
server_key_iv: "910067664ec431c51b6612ae5ba803de"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.105453 4729 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:42:04.107856 4747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.109095 4729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:42:04.109246 4729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "662782d9a7cf46558b4f73392916c5e8"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "95263a6f4e3d7697cb27aaca6949262b"
server_key_iv: "910067664ec431c51b6612ae5ba803de"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.109362 4729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:04.131361 4729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:04.134516 4729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:04.134709 4729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:04.135320 4729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:04.136164 4729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:42:04.136230 4729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.136300 4729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:42:04.136348 4729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.145578 4729 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:35005
I20260810 22:42:04.145612 4869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:35005 every 8 connection(s)
I20260810 22:42:04.146531 4729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:42:04.154500 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 4729
I20260810 22:42:04.154645 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:42:04.154982 20265 external_mini_cluster.cc:1428] Setting key bf0c104564175cbde10d80e043630c01
Aug 10 22:42:04 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401724, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:42:04.160447 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.148212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:51223 (local address 127.19.202.126:37125)
0810 22:42:04.148367 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.148370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.149119 (+ 749us) server_negotiation.cc:407] Connection header received
0810 22:42:04.150430 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.150433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.150491 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.150586 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:04.151672 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.152176 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.153304 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.153553 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.153662 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.156663 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:04.156666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:04.156668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:04.156691 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:04.158199 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.158797 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.158803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.158807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.158909 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.159224 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.159230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.159233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.159493 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:04.159595 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.160057 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.160197 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:42:04.161078 4872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.148553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:51223)
0810 22:42:04.148976 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.149017 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.150230 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.150745 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.150751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.151061 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:04.151527 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.151540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.152296 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.152299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.153186 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.153195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.153701 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.153705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.153871 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.154733 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:04.154754 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:04.156483 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:04.158319 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:04.158327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:04.158345 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:04.158700 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:04.159006 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:04.159008 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:04.159009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:04.159138 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:04.159599 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:04.159604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:04.159813 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.159981 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.160212 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":108,"threads_started":1}
I20260810 22:42:04.162393 4870 heartbeater.cc:347] Connected to a master server at 127.19.202.126:37125
I20260810 22:42:04.162643 4870 heartbeater.cc:464] Registering TS with master...
I20260810 22:42:04.163110 4870 heartbeater.cc:511] Master 127.19.202.126:37125 requested a full tablet report, sending...
I20260810 22:42:04.164499 4660 ts_manager.cc:194] Registered new tserver with Master: 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005)
I20260810 22:42:04.165668 4660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:51223
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:42:04.181847 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:37125
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:42:04.293135 4875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:04.293376 4875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:04.293421 4875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:04.297362 4875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:04.297415 4875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:04.297474 4875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:42:04.302538 4875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:04.303460 4875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:04.304420 4875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:04.313792 4881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:04.313832 4880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:04.314033 4875 server_base.cc:1034] running on GCE node
W20260810 22:42:04.314031 4883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:04.314518 4875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:04.315145 4875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:04.316329 4875 hybrid_clock.cc:648] HybridClock initialized: now 1786401724316305 us; error 50 us; skew 500 ppm
Aug 10 22:42:04 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401724, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:04.319057 4875 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:42:04.319883 4875 webserver.cc:460] Webserver started at http://127.19.202.66:41359/ using document root <none> and password file <none>
I20260810 22:42:04.320459 4875 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:04.320547 4875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:04.320770 4875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:42:04.322821 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "77a1b3b45fa74f888f04b51b12aa548d"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a2cf561d82ae9742960df0a18c8127f"
server_key_iv: "af024af9f91f4c35181fe4aa1532467b"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.323467 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "77a1b3b45fa74f888f04b51b12aa548d"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a2cf561d82ae9742960df0a18c8127f"
server_key_iv: "af024af9f91f4c35181fe4aa1532467b"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.327371 4875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 22:42:04.330094 4890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.331503 4875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:42:04.331655 4875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "77a1b3b45fa74f888f04b51b12aa548d"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a2cf561d82ae9742960df0a18c8127f"
server_key_iv: "af024af9f91f4c35181fe4aa1532467b"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.331777 4875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:04.351840 4875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:04.354727 4875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:04.354944 4875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:04.355546 4875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:04.356397 4875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:42:04.356468 4875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.356541 4875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:42:04.356612 4875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.366394 4875 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:35077
I20260810 22:42:04.366413 5015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:35077 every 8 connection(s)
I20260810 22:42:04.367280 4875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:42:04.368234 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 4875
I20260810 22:42:04.368386 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:42:04.368713 20265 external_mini_cluster.cc:1428] Setting key 2006df4bf200c35e034af52032e23855
Aug 10 22:42:04 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401724, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:42:04.382805 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.369437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:45233 (local address 127.19.202.126:37125)
0810 22:42:04.369576 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.369579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.370530 (+ 951us) server_negotiation.cc:407] Connection header received
0810 22:42:04.371882 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.371886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.371941 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.372020 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:04.373455 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.373971 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.375798 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.376016 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.376100 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.379001 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:04.379004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:04.379006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:04.379034 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:04.380524 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.381132 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.381136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.381138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.381234 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.381660 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.381663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.381664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.381846 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:04.381953 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.382481 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.382646 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
I20260810 22:42:04.383620 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.369846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:45233)
0810 22:42:04.370379 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.370441 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.371681 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.372182 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.372189 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.372560 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:04.373299 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.373320 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.374102 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.374108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.375672 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.375688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.376172 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.376177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.376409 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.377085 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:04.377111 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:04.378826 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:04.380635 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:04.380641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:04.380654 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:04.381026 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:04.381328 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:04.381331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:04.381333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:04.381553 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:04.381954 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:04.381962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:04.382204 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.382382 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.382702 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":151,"threads_started":1}
I20260810 22:42:04.384799 5016 heartbeater.cc:347] Connected to a master server at 127.19.202.126:37125
I20260810 22:42:04.385049 5016 heartbeater.cc:464] Registering TS with master...
I20260810 22:42:04.385543 5016 heartbeater.cc:511] Master 127.19.202.126:37125 requested a full tablet report, sending...
I20260810 22:42:04.386613 4660 ts_manager.cc:194] Registered new tserver with Master: 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077)
I20260810 22:42:04.387185 4660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:45233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:42:04.399145 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:37125
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:42:04.510794 5022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:04.511030 5022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:04.511067 5022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:04.514968 5022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:04.515017 5022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:04.515072 5022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:42:04.520004 5022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:04.520958 5022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:04.521848 5022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:04.531237 5028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:04.531399 5022 server_base.cc:1034] running on GCE node
W20260810 22:42:04.531282 5029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:04.531273 5031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:04.531978 5022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:04.532591 5022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:04.533746 5022 hybrid_clock.cc:648] HybridClock initialized: now 1786401724533748 us; error 45 us; skew 500 ppm
Aug 10 22:42:04 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401724, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:04.536669 5022 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:42:04.537580 5022 webserver.cc:460] Webserver started at http://127.19.202.67:39913/ using document root <none> and password file <none>
I20260810 22:42:04.538092 5022 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:04.538153 5022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:04.538316 5022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:42:04.540271 5022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "5eee40525a8f4af48f2568dbbe6f014e"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a620bc255d70c3642229d60ddaa256a"
server_key_iv: "357857e7546c8c7d127e6f94c4fd2bb3"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.540853 5022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "5eee40525a8f4af48f2568dbbe6f014e"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a620bc255d70c3642229d60ddaa256a"
server_key_iv: "357857e7546c8c7d127e6f94c4fd2bb3"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.544385 5022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:42:04.546829 5039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.548022 5022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:42:04.548178 5022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "5eee40525a8f4af48f2568dbbe6f014e"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a620bc255d70c3642229d60ddaa256a"
server_key_iv: "357857e7546c8c7d127e6f94c4fd2bb3"
server_key_version: "encryptionkey@0"
I20260810 22:42:04.548282 5022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:04.570658 5022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:04.573798 5022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:04.573987 5022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:04.574525 5022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:04.575418 5022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:42:04.575491 5022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.575553 5022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:42:04.575598 5022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:04.584435 5022 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:44741
I20260810 22:42:04.584461 5164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:44741 every 8 connection(s)
I20260810 22:42:04.585294 5022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:42:04.585747 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 5022
I20260810 22:42:04.585906 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:42:04.586242 20265 external_mini_cluster.cc:1428] Setting key 204821e87ffd261c6808b74af7800f40
Aug 10 22:42:04 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401724, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:42:04.598330 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.587103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:57935 (local address 127.19.202.126:37125)
0810 22:42:04.587216 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.587220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.588272 (+ 1052us) server_negotiation.cc:407] Connection header received
0810 22:42:04.589663 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.589666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.589712 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.589780 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:04.590855 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.591433 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.592680 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.592954 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.593028 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.595348 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:04.595351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:04.595352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:04.595378 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:04.596602 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.597105 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.597108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.597109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.597187 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.597469 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.597471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.597472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.597630 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:04.597704 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.598134 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.598224 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 22:42:04.599097 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.587573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:57935)
0810 22:42:04.588090 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.588143 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.589484 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.589908 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.589914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.590253 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:04.590744 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.590756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.591594 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.591598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.592581 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.592591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.593076 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.593079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.593218 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.593676 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:04.593690 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:04.595175 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:04.596705 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:04.596709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:04.596719 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:04.597023 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:04.597267 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:04.597269 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:04.597270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:04.597395 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:04.597713 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:04.597718 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:04.597917 (+ 199us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.598092 (+ 175us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.598299 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":103,"threads_started":1}
I20260810 22:42:04.600088 5165 heartbeater.cc:347] Connected to a master server at 127.19.202.126:37125
I20260810 22:42:04.600315 5165 heartbeater.cc:464] Registering TS with master...
I20260810 22:42:04.600751 5165 heartbeater.cc:511] Master 127.19.202.126:37125 requested a full tablet report, sending...
I20260810 22:42:04.601742 4660 ts_manager.cc:194] Registered new tserver with Master: 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
I20260810 22:42:04.602279 4660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:57935
I20260810 22:42:04.611704 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:42:04.620406 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.613840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.19.202.126:37125)
0810 22:42:04.613960 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.613963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.614169 (+ 206us) server_negotiation.cc:407] Connection header received
0810 22:42:04.614244 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.614246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.614279 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.614382 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:04.615014 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.615509 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.616269 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.616466 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.616543 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.617817 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:04.617820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:04.617822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:04.617842 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:04.619075 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.619447 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.619449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.619450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.619519 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:04.619737 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:04.619739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:04.619740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:04.619938 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:04.620042 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.620222 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.620304 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:42:04.623099 4660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:42:04.626286 4660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:42:04.640803 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.634809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:44741 (local address 127.0.0.1:47136)
0810 22:42:04.635677 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.635691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.635795 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.636488 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.636494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.636519 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:04.636651 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.636657 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.638092 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.638096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.639415 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.639427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.640237 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.640240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.640342 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.640380 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.640495 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.640618 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":6,"thread_start_us":80,"threads_started":1}
I20260810 22:42:04.640988 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.634578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:35077 (local address 127.0.0.1:52550)
0810 22:42:04.635241 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.635255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.635347 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.636562 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.636583 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.636596 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:04.636661 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.636665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.637806 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.637811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.639415 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.639431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.640449 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.640453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.640568 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.640657 (+ 89us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.640724 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.640844 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":270,"thread_start_us":100,"threads_started":1}
I20260810 22:42:04.641321 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.635204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.19.202.66:35077)
0810 22:42:04.636056 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.636068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.636088 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:42:04.636223 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.636228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.636412 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.636847 (+ 435us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:04.636993 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.637681 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.639551 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.640306 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.640663 (+ 357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.640836 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.640887 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.641184 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":153,"threads_started":1}
I20260810 22:42:04.641359 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.635239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47136 (local address 127.19.202.67:44741)
0810 22:42:04.635686 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.635699 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.635784 (+ 85us) server_negotiation.cc:407] Connection header received
0810 22:42:04.635993 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.636001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.636290 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.636489 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:04.636759 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.637942 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.639577 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.640110 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.640224 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.640369 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.641138 (+ 769us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.641226 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":198,"threads_started":1}
I20260810 22:42:04.641358 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.634551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.19.202.65:35005)
0810 22:42:04.635763 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.635779 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.635802 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:42:04.635885 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.635892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.636106 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.636259 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:04.636679 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.637938 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.639399 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.640250 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.640376 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.640527 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.640880 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.641210 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":134,"threads_started":1}
I20260810 22:42:04.641606 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.634384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35005 (local address 127.0.0.1:47650)
0810 22:42:04.635046 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.635097 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.635227 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.636310 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.636316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.636354 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:04.636507 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.636514 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.638140 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.638144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.639269 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.639279 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.640450 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.640457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.640569 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.640622 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.640687 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.640738 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":37,"thread_start_us":96,"threads_started":1}
I20260810 22:42:04.644032 4945 tablet_service.cc:1467] Processing CreateTablet for tablet dc277ae7b0564cd7b57d162e089b6153 (DEFAULT_TABLE table=test-table [id=63ac28542df74205aacfc9c733a1d887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:42:04.644160 5093 tablet_service.cc:1467] Processing CreateTablet for tablet dc277ae7b0564cd7b57d162e089b6153 (DEFAULT_TABLE table=test-table [id=63ac28542df74205aacfc9c733a1d887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:42:04.644454 4798 tablet_service.cc:1467] Processing CreateTablet for tablet dc277ae7b0564cd7b57d162e089b6153 (DEFAULT_TABLE table=test-table [id=63ac28542df74205aacfc9c733a1d887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:42:04.645143 4945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc277ae7b0564cd7b57d162e089b6153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:42:04.645244 5093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc277ae7b0564cd7b57d162e089b6153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:42:04.645352 4798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc277ae7b0564cd7b57d162e089b6153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:42:04.651759 5184 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Bootstrap starting.
I20260810 22:42:04.652917 5187 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Bootstrap starting.
I20260810 22:42:04.654587 5187 tablet_bootstrap.cc:654] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Neither blocks nor log segments found. Creating new log.
I20260810 22:42:04.654857 5184 tablet_bootstrap.cc:654] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Neither blocks nor log segments found. Creating new log.
I20260810 22:42:04.655098 5186 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Bootstrap starting.
I20260810 22:42:04.655644 5184 log.cc:824] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:04.655862 5187 log.cc:824] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:04.657225 5186 tablet_bootstrap.cc:654] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Neither blocks nor log segments found. Creating new log.
I20260810 22:42:04.657760 5187 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: No bootstrap required, opened a new log
I20260810 22:42:04.657759 5184 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: No bootstrap required, opened a new log
I20260810 22:42:04.657956 5187 ts_tablet_manager.cc:1397] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:42:04.657956 5184 ts_tablet_manager.cc:1397] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 22:42:04.658315 5186 log.cc:824] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:04.659819 5186 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: No bootstrap required, opened a new log
I20260810 22:42:04.660010 5186 ts_tablet_manager.cc:1397] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:42:04.661783 5187 raft_consensus.cc:348] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.661783 5184 raft_consensus.cc:348] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.662055 5184 raft_consensus.cc:374] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:42:04.662055 5187 raft_consensus.cc:374] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:42:04.662137 5187 raft_consensus.cc:729] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662782d9a7cf46558b4f73392916c5e8, State: Initialized, Role: FOLLOWER
I20260810 22:42:04.662163 5184 raft_consensus.cc:729] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a1b3b45fa74f888f04b51b12aa548d, State: Initialized, Role: FOLLOWER
I20260810 22:42:04.662626 5187 consensus_queue.cc:260] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.662626 5184 consensus_queue.cc:260] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.663100 5186 raft_consensus.cc:348] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.663420 5186 raft_consensus.cc:374] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:42:04.663548 5186 raft_consensus.cc:729] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eee40525a8f4af48f2568dbbe6f014e, State: Initialized, Role: FOLLOWER
I20260810 22:42:04.663601 5187 ts_tablet_manager.cc:1428] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:42:04.663600 5184 ts_tablet_manager.cc:1428] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:42:04.663959 4870 heartbeater.cc:503] Master 127.19.202.126:37125 was elected leader, sending a full tablet report...
I20260810 22:42:04.664057 5191 raft_consensus.cc:482] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:04.664040 5186 consensus_queue.cc:260] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.664158 5191 raft_consensus.cc:504] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.664548 5016 heartbeater.cc:503] Master 127.19.202.126:37125 was elected leader, sending a full tablet report...
I20260810 22:42:04.664999 5186 ts_tablet_manager.cc:1428] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:42:04.665060 5165 heartbeater.cc:503] Master 127.19.202.126:37125 was elected leader, sending a full tablet report...
I20260810 22:42:04.667074 5191 leader_election.cc:288] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
I20260810 22:42:04.670118 5194 raft_consensus.cc:482] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:04.670264 5194 raft_consensus.cc:504] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.671195 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.666328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:35077 (local address 127.19.202.65:37531)
0810 22:42:04.666771 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.666787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.666926 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.667340 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.667344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.667362 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:04.667492 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.667499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.668481 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.668484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.669738 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.669749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.670740 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.670744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.670844 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.670882 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.670972 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.671047 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":114,"threads_started":1}
I20260810 22:42:04.671289 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.667185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:52193 (local address 127.19.202.67:44741)
0810 22:42:04.667308 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.667313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.667323 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:42:04.667359 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.667362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.667395 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.667478 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:04.667807 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.668499 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.670013 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.670605 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.670719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.670769 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.671108 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.671178 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:42:04.671310 5194 leader_election.cc:288] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005), 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077)
I20260810 22:42:04.671535 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.666561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:37531 (local address 127.19.202.66:35077)
0810 22:42:04.666693 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.666697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.666854 (+ 157us) server_negotiation.cc:407] Connection header received
0810 22:42:04.667060 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.667064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.667117 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.667198 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:04.667613 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.668387 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.669880 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.670480 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.670614 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.670664 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.671347 (+ 683us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.671437 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:42:04.671747 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.666854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:44741 (local address 127.19.202.65:52193)
0810 22:42:04.667131 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.667155 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.667256 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.667578 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.667583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.667599 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:04.667691 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.667696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.668630 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.668632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.669892 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.669902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.670845 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.670849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.670963 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.670988 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.671527 (+ 539us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.671597 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":104,"threads_started":1}
I20260810 22:42:04.672345 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc277ae7b0564cd7b57d162e089b6153" candidate_uuid: "662782d9a7cf46558b4f73392916c5e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77a1b3b45fa74f888f04b51b12aa548d" is_pre_election: true
I20260810 22:42:04.672588 4967 raft_consensus.cc:2454] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 662782d9a7cf46558b4f73392916c5e8 in term 0.
I20260810 22:42:04.672585 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc277ae7b0564cd7b57d162e089b6153" candidate_uuid: "662782d9a7cf46558b4f73392916c5e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eee40525a8f4af48f2568dbbe6f014e" is_pre_election: true
I20260810 22:42:04.672745 5114 raft_consensus.cc:2454] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 662782d9a7cf46558b4f73392916c5e8 in term 0.
I20260810 22:42:04.673059 4750 leader_election.cc:302] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8, 77a1b3b45fa74f888f04b51b12aa548d; no voters:
I20260810 22:42:04.673517 5191 raft_consensus.cc:2781] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:42:04.673615 5191 raft_consensus.cc:482] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:42:04.673767 5191 raft_consensus.cc:3037] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:42:04.674844 5191 raft_consensus.cc:504] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.675457 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc277ae7b0564cd7b57d162e089b6153" candidate_uuid: "662782d9a7cf46558b4f73392916c5e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77a1b3b45fa74f888f04b51b12aa548d"
I20260810 22:42:04.675560 4967 raft_consensus.cc:3037] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:42:04.675592 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.671931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:43421 (local address 127.19.202.65:35005)
0810 22:42:04.672036 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.672040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.672051 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:42:04.672089 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.672091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.672120 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.672200 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:04.672448 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.673093 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.674546 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.675038 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.675160 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.675221 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.675411 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.675475 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:42:04.675585 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.671471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35005 (local address 127.19.202.67:43421)
0810 22:42:04.671676 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.671688 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.671754 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.672212 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.672215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.672231 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:04.672329 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.672335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.673204 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.673207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.674410 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.674421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.675159 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.675163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.675289 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.675313 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.675397 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.675469 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":32}
I20260810 22:42:04.676069 4820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc277ae7b0564cd7b57d162e089b6153" candidate_uuid: "5eee40525a8f4af48f2568dbbe6f014e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662782d9a7cf46558b4f73392916c5e8" is_pre_election: true
I20260810 22:42:04.676277 4820 raft_consensus.cc:2379] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5eee40525a8f4af48f2568dbbe6f014e in current term 1: Already voted for candidate 662782d9a7cf46558b4f73392916c5e8 in this term.
I20260810 22:42:04.676702 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc277ae7b0564cd7b57d162e089b6153" candidate_uuid: "662782d9a7cf46558b4f73392916c5e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eee40525a8f4af48f2568dbbe6f014e"
I20260810 22:42:04.676853 5114 raft_consensus.cc:3037] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:42:04.676972 4967 raft_consensus.cc:2454] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 662782d9a7cf46558b4f73392916c5e8 in term 1.
I20260810 22:42:04.677253 4750 leader_election.cc:302] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8, 77a1b3b45fa74f888f04b51b12aa548d; no voters:
I20260810 22:42:04.677441 5193 raft_consensus.cc:2781] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:42:04.677626 5193 raft_consensus.cc:686] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 LEADER]: Becoming Leader. State: Replica: 662782d9a7cf46558b4f73392916c5e8, State: Running, Role: LEADER
I20260810 22:42:04.677639 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.672158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:52615 (local address 127.19.202.66:35077)
0810 22:42:04.672769 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.672792 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.673005 (+ 213us) server_negotiation.cc:407] Connection header received
0810 22:42:04.673178 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.673181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.673220 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.673265 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:04.673915 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.674579 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.676008 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.676554 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.676713 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.676753 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.677474 (+ 721us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.677536 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545}
I20260810 22:42:04.677685 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.671577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:35077 (local address 127.19.202.67:52615)
0810 22:42:04.672930 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.672944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:04.673065 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:04.673402 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:04.673405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:04.673421 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:04.673799 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.673811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.674698 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.674704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.675844 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:04.675856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.677200 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.677204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:04.677301 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.677325 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 22:42:04.677453 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 22:42:04.677536 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1290,"mutex_wait_us":181,"thread_start_us":102,"threads_started":1}
I20260810 22:42:04.677937 5193 consensus_queue.cc:237] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:04.678123 5114 raft_consensus.cc:2454] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 662782d9a7cf46558b4f73392916c5e8 in term 1.
I20260810 22:42:04.678483 5191 leader_election.cc:288] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 1 election: Requested vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
I20260810 22:42:04.678669 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc277ae7b0564cd7b57d162e089b6153" candidate_uuid: "5eee40525a8f4af48f2568dbbe6f014e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77a1b3b45fa74f888f04b51b12aa548d" is_pre_election: true
I20260810 22:42:04.678812 4967 raft_consensus.cc:2379] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5eee40525a8f4af48f2568dbbe6f014e in current term 1: Already voted for candidate 662782d9a7cf46558b4f73392916c5e8 in this term.
I20260810 22:42:04.679174 5042 leader_election.cc:302] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5eee40525a8f4af48f2568dbbe6f014e; no voters: 662782d9a7cf46558b4f73392916c5e8, 77a1b3b45fa74f888f04b51b12aa548d
I20260810 22:42:04.679421 5194 raft_consensus.cc:2726] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 22:42:04.681195 4659 catalog_manager.cc:5495] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "662782d9a7cf46558b4f73392916c5e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } health_report { overall_health: UNKNOWN } } }
I20260810 22:42:04.691174 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.687827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.19.202.65:35005)
0810 22:42:04.687935 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:04.687939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:04.687982 (+ 43us) server_negotiation.cc:407] Connection header received
0810 22:42:04.688125 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:04.688127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:04.688171 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:04.688250 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:42:04.688485 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.689036 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.689826 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:04.690023 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:04.690094 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:04.690382 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:42:04.690737 (+ 355us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:42:04.690936 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:04.690980 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 22:42:04.691023 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:42:04.696367 4967 raft_consensus.cc:1261] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Refusing update from remote peer 662782d9a7cf46558b4f73392916c5e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:42:04.696413 5114 raft_consensus.cc:1261] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Refusing update from remote peer 662782d9a7cf46558b4f73392916c5e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:42:04.697124 5198 consensus_queue.cc:1035] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:42:04.697311 5193 consensus_queue.cc:1035] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:42:04.704859 5201 mvcc.cc:205] Tried to move back new op lower bound from 7317101464350425088 to 7317101464284053504. Current Snapshot: MvccSnapshot[applied={T|T < 7317101464350425088}]
I20260810 22:42:04.713814 4659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:42:04.713975 4659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:42:04.716300 4659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Sending DeleteTablet for 3 replicas of tablet dc277ae7b0564cd7b57d162e089b6153
I20260810 22:42:04.717090 4798 tablet_service.cc:1514] Processing DeleteTablet for tablet dc277ae7b0564cd7b57d162e089b6153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:42:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:47650
I20260810 22:42:04.717094 4945 tablet_service.cc:1514] Processing DeleteTablet for tablet dc277ae7b0564cd7b57d162e089b6153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:42:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:52550
I20260810 22:42:04.717585 5211 tablet_replica.cc:330] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: stopping tablet replica
I20260810 22:42:04.717577 5093 tablet_service.cc:1514] Processing DeleteTablet for tablet dc277ae7b0564cd7b57d162e089b6153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:42:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:47136
I20260810 22:42:04.717991 5212 tablet_replica.cc:330] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: stopping tablet replica
I20260810 22:42:04.717995 5211 raft_consensus.cc:2229] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:42:04.718219 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 4729
I20260810 22:42:04.718299 5211 raft_consensus.cc:2258] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:42:04.718349 5212 raft_consensus.cc:2229] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:42:04.720970 5211 ts_tablet_manager.cc:1905] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:42:04.723845 5211 ts_tablet_manager.cc:1918] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:42:04.723973 5211 log.cc:1196] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/dc277ae7b0564cd7b57d162e089b6153
I20260810 22:42:04.724298 5211 ts_tablet_manager.cc:1939] T dc277ae7b0564cd7b57d162e089b6153 P 77a1b3b45fa74f888f04b51b12aa548d: Deleting consensus metadata
I20260810 22:42:04.725425 4644 catalog_manager.cc:4841] TS 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): tablet dc277ae7b0564cd7b57d162e089b6153 (table test-table [id=63ac28542df74205aacfc9c733a1d887]) successfully deleted
I20260810 22:42:04.725483 5212 raft_consensus.cc:2258] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:42:04.726701 5212 ts_tablet_manager.cc:1905] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 22:42:04.728863 4645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:35005 (error 108)
I20260810 22:42:04.729089 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 4875
I20260810 22:42:04.729882 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:04.729588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35005 (local address 127.0.0.1:47664)
0810 22:42:04.729757 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:04.729813 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260810 22:42:04.730068 4645 catalog_manager.cc:4584] TS 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc277ae7b0564cd7b57d162e089b6153: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35005: connect: Connection refused (error 111)
I20260810 22:42:04.730772 5212 ts_tablet_manager.cc:1918] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:42:04.730880 5212 log.cc:1196] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/dc277ae7b0564cd7b57d162e089b6153
I20260810 22:42:04.731449 5212 ts_tablet_manager.cc:1939] T dc277ae7b0564cd7b57d162e089b6153 P 5eee40525a8f4af48f2568dbbe6f014e: Deleting consensus metadata
I20260810 22:42:04.732438 4643 catalog_manager.cc:4841] TS 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): tablet dc277ae7b0564cd7b57d162e089b6153 (table test-table [id=63ac28542df74205aacfc9c733a1d887]) successfully deleted
I20260810 22:42:04.737074 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 5022
I20260810 22:42:04.743983 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 4626
I20260810 22:42:24.753464 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37125
--webserver_interface=127.19.202.126
--webserver_port=36175
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:42:24.866555 5214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:24.866822 5214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:24.866864 5214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:24.870757 5214 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 22:42:24.870806 5214 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 22:42:24.870824 5214 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 22:42:24.870841 5214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:42:24.870860 5214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:24.870880 5214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:24.870905 5214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:42:24.870930 5214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:42:24.875846 5214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=36175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:24.876766 5214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:24.877760 5214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:24.891978 5222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:24.891916 5221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:24.891935 5224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:24.892004 5214 server_base.cc:1034] running on GCE node
I20260810 22:42:24.892881 5214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:24.893920 5214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:24.895112 5214 hybrid_clock.cc:648] HybridClock initialized: now 1786401744895091 us; error 31 us; skew 500 ppm
Aug 10 22:42:24 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401744, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:24.898160 5214 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:42:24.899014 5214 webserver.cc:460] Webserver started at http://127.19.202.126:36175/ using document root <none> and password file <none>
I20260810 22:42:24.899575 5214 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:24.899670 5214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:24.903416 5214 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:42:24.905480 5231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:24.906620 5214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:42:24.906781 5214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "e8593e37e2dd419299520249db4b89da"
format_stamp: "Formatted at 2026-08-10 22:42:03 on dist-test-slave-1x4g"
server_key: "6d1c9467486ee970c2899d6d60b02b62"
server_key_iv: "c874adf321bcd5099179811704690dcc"
server_key_version: "encryptionkey@0"
I20260810 22:42:24.907400 5214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:24.920713 5214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:24.923914 5214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:24.924109 5214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:24.931124 5214 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:37125
I20260810 22:42:24.931399 5287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:37125 every 8 connection(s)
I20260810 22:42:24.932207 5214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:42:24.936514 5288 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:42:24.938195 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Bootstrap starting.
I20260810 22:42:24.940879 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 5214
I20260810 22:42:24.941401 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:35005
--local_ip_for_outbound_sockets=127.19.202.65
--tserver_master_addrs=127.19.202.126:37125
--webserver_port=41875
--webserver_interface=127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:42:24.956322 5288 log.cc:824] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:24.964735 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:42:24.965135 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Bootstrap complete.
I20260810 22:42:24.969298 5288 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:24.969688 5288 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8593e37e2dd419299520249db4b89da, State: Initialized, Role: FOLLOWER
I20260810 22:42:24.970152 5288 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:24.970261 5288 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:42:24.970355 5288 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:42:24.970467 5288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:42:24.972304 5288 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:24.972632 5288 leader_election.cc:302] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8593e37e2dd419299520249db4b89da; no voters:
I20260810 22:42:24.973052 5288 leader_election.cc:288] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 22:42:24.973129 5293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 FOLLOWER]: Leader election won for term 2
I20260810 22:42:24.973328 5293 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 LEADER]: Becoming Leader. State: Replica: e8593e37e2dd419299520249db4b89da, State: Running, Role: LEADER
I20260810 22:42:24.973665 5293 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:24.974273 5288 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:42:24.974612 5293 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8593e37e2dd419299520249db4b89da. Latest consensus state: current_term: 2 leader_uuid: "e8593e37e2dd419299520249db4b89da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } } }
I20260810 22:42:24.974730 5293 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: This master's current role is: LEADER
I20260810 22:42:24.975083 5298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:42:24.975109 5295 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e8593e37e2dd419299520249db4b89da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } } }
I20260810 22:42:24.975287 5295 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: This master's current role is: LEADER
I20260810 22:42:24.979979 5298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:42:24.980634 5298 catalog_manager.cc:1259] Loaded cluster ID: 595527617bda4cc888db28562993bf3c
I20260810 22:42:24.980702 5298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:42:24.983187 5298 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:42:24.983863 5298 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Loaded TSK: 0
I20260810 22:42:25.000180 5298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Generated new TSK 1
I20260810 22:42:25.000422 5298 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 22:42:25.056427 5290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:25.056701 5290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:25.056823 5290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:25.060814 5290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:25.060890 5290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:25.060972 5290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:42:25.065963 5290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:35005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=41875
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:25.066964 5290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:25.068005 5290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:25.077693 5319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:25.077708 5318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:25.078045 5322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:25.078390 5290 server_base.cc:1034] running on GCE node
I20260810 22:42:25.078915 5290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:25.079543 5290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:25.080757 5290 hybrid_clock.cc:648] HybridClock initialized: now 1786401745080719 us; error 53 us; skew 500 ppm
Aug 10 22:42:25 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401745, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:25.083818 5290 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:42:25.084883 5290 webserver.cc:460] Webserver started at http://127.19.202.65:41875/ using document root <none> and password file <none>
I20260810 22:42:25.085631 5290 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:25.085733 5290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:25.090720 5290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260810 22:42:25.093462 5330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:25.095045 5290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 22:42:25.095168 5290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "662782d9a7cf46558b4f73392916c5e8"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "95263a6f4e3d7697cb27aaca6949262b"
server_key_iv: "910067664ec431c51b6612ae5ba803de"
server_key_version: "encryptionkey@0"
I20260810 22:42:25.095867 5290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:25.110819 5290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:25.114416 5290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:25.114605 5290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:25.115305 5290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:25.116607 5339 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 22:42:25.120977 5290 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 22:42:25.121049 5290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 22:42:25.121112 5290 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 22:42:25.123322 5290 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 22:42:25.123387 5290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 22:42:25.123709 5339 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Bootstrap starting.
I20260810 22:42:25.133884 5290 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:35005
I20260810 22:42:25.134232 5456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:35005 every 8 connection(s)
I20260810 22:42:25.135149 5290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:42:25.138836 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 5290
I20260810 22:42:25.139425 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:35077
--local_ip_for_outbound_sockets=127.19.202.66
--tserver_master_addrs=127.19.202.126:37125
--webserver_port=41359
--webserver_interface=127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:42:25.147822 5339 log.cc:824] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Log is configured to *not* fsync() on all Append() calls
Aug 10 22:42:25 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401745, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:42:25.156339 5339 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:42:25.156927 5339 tablet_bootstrap.cc:492] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Bootstrap complete.
I20260810 22:42:25.157250 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.137370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:36763)
0810 22:42:25.141138 (+ 3768us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.141201 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.142497 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.144023 (+ 1526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.144029 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.144410 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:25.144957 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.144974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.146260 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.146263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.147208 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.147218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.148018 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.148023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.148234 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.148988 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:25.149008 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:25.151022 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:25.153703 (+ 2681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.153712 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.153726 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.154121 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.154555 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.154560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.154563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.154749 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.155404 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:25.155410 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:25.155717 (+ 307us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.155903 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.156219 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3509,"thread_start_us":152,"threads_started":1}
I20260810 22:42:25.157766 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.137344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:36763 (local address 127.19.202.126:37125)
0810 22:42:25.138973 (+ 1629us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.139021 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.141644 (+ 2623us) server_negotiation.cc:407] Connection header received
0810 22:42:25.143038 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.143060 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.143544 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.143965 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:25.145137 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.146115 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.147339 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.147792 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.147985 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.151259 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:25.151266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:25.151280 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:25.151315 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:25.153552 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.154271 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.154277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.154282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.154444 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.154858 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.154861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.154862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.155280 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:25.155420 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.156067 (+ 647us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.156376 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1364,"thread_start_us":116,"threads_started":1}
I20260810 22:42:25.158550 5339 ts_tablet_manager.cc:1397] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Time spent bootstrapping tablet: real 0.035s user 0.027s sys 0.004s
I20260810 22:42:25.160004 5457 heartbeater.cc:347] Connected to a master server at 127.19.202.126:37125
I20260810 22:42:25.160228 5457 heartbeater.cc:464] Registering TS with master...
I20260810 22:42:25.160749 5457 heartbeater.cc:511] Master 127.19.202.126:37125 requested a full tablet report, sending...
I20260810 22:42:25.161211 5339 raft_consensus.cc:348] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:25.161686 5339 raft_consensus.cc:729] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662782d9a7cf46558b4f73392916c5e8, State: Initialized, Role: FOLLOWER
I20260810 22:42:25.162115 5339 consensus_queue.cc:260] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } } peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } }
I20260810 22:42:25.163024 5339 ts_tablet_manager.cc:1428] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260810 22:42:25.163339 5250 ts_manager.cc:194] Registered new tserver with Master: 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005)
I20260810 22:42:25.167614 5250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:36763
I20260810 22:42:25.170094 5457 heartbeater.cc:503] Master 127.19.202.126:37125 was elected leader, sending a full tablet report...
Aug 10 22:42:25 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401744, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.65@KRBTEST.COM
I20260810 22:42:25.177197 5465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.166599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54446 (local address 127.19.202.65:35005)
0810 22:42:25.167373 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.167381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.167399 (+ 18us) server_negotiation.cc:407] Connection header received
0810 22:42:25.167447 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.167450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.167588 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.167792 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:25.168358 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.169371 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.170564 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.170978 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.171086 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.173556 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:25.173564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:25.173573 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:25.173593 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:25.175186 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.175782 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.175788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.175790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.175895 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.176185 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.176188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.176189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.176407 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:25.176507 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.176949 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.177081 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":127,"threads_started":1}
I20260810 22:42:25.177191 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.166557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35005 (local address 127.0.0.1:54446)
0810 22:42:25.166949 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.166992 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.167106 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.167733 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.167741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.168136 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:25.168233 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.168239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.169503 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.169507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.170427 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.170439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.171100 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.171104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.171246 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.171804 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:25.171816 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:25.173422 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:25.175302 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.175313 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.175320 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.175670 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.175996 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.175999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.176001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.176095 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.176516 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:25.176521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:25.176674 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.176846 (+ 172us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.177031 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":131,"threads_started":1}
I20260810 22:42:25.178321 5384 tablet_service.cc:1514] Processing DeleteTablet for tablet dc277ae7b0564cd7b57d162e089b6153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:42:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:54446
I20260810 22:42:25.179126 5466 tablet_replica.cc:330] stopping tablet replica
I20260810 22:42:25.179402 5466 raft_consensus.cc:2229] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:42:25.179637 5466 raft_consensus.cc:2258] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:42:25.180524 5466 ts_tablet_manager.cc:1905] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:42:25.184028 5466 ts_tablet_manager.cc:1918] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:42:25.184108 5466 log.cc:1196] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal/wals/dc277ae7b0564cd7b57d162e089b6153
I20260810 22:42:25.184370 5466 ts_tablet_manager.cc:1939] T dc277ae7b0564cd7b57d162e089b6153 P 662782d9a7cf46558b4f73392916c5e8: Deleting consensus metadata
I20260810 22:42:25.185935 5235 catalog_manager.cc:4841] TS 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005): tablet dc277ae7b0564cd7b57d162e089b6153 (table test-table [id=63ac28542df74205aacfc9c733a1d887]) successfully deleted
W20260810 22:42:25.262197 5461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:25.262488 5461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:25.262578 5461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:25.266618 5461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:25.266708 5461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:25.266798 5461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:42:25.271718 5461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:35077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=41359
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:25.272676 5461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:25.273772 5461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:25.283382 5472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:25.283370 5475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:25.283365 5473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:25.283732 5461 server_base.cc:1034] running on GCE node
I20260810 22:42:25.284090 5461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:25.284659 5461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:25.285815 5461 hybrid_clock.cc:648] HybridClock initialized: now 1786401745285802 us; error 30 us; skew 500 ppm
Aug 10 22:42:25 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401745, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:25.288431 5461 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:42:25.289327 5461 webserver.cc:460] Webserver started at http://127.19.202.66:41359/ using document root <none> and password file <none>
I20260810 22:42:25.289880 5461 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:25.289983 5461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:25.293740 5461 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:42:25.295785 5482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:25.297080 5461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:42:25.297215 5461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "77a1b3b45fa74f888f04b51b12aa548d"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a2cf561d82ae9742960df0a18c8127f"
server_key_iv: "af024af9f91f4c35181fe4aa1532467b"
server_key_version: "encryptionkey@0"
I20260810 22:42:25.297749 5461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:25.314239 5461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:25.317390 5461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:25.317581 5461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:25.318156 5461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:25.318940 5461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:42:25.318987 5461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:25.319048 5461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:42:25.319080 5461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:25.327932 5461 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:35077
I20260810 22:42:25.328004 5612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:35077 every 8 connection(s)
I20260810 22:42:25.328974 5461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:42:25.329514 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 5461
I20260810 22:42:25.329991 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:44741
--local_ip_for_outbound_sockets=127.19.202.67
--tserver_master_addrs=127.19.202.126:37125
--webserver_port=39913
--webserver_interface=127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:42:25 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401745, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:42:25.344811 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.330953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:33569 (local address 127.19.202.126:37125)
0810 22:42:25.331092 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.331096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.332211 (+ 1115us) server_negotiation.cc:407] Connection header received
0810 22:42:25.334095 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.334101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.334168 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.334263 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:25.335583 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.336609 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.337828 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.338288 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.338433 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.340864 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:25.340870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:25.340874 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:25.340906 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:25.342653 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.343169 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.343173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.343175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.343315 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.343683 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.343686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.343688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.343910 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:25.344009 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.344482 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.344620 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:42:25.345723 5615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.331441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:33569)
0810 22:42:25.332024 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.332086 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.333866 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.334437 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.334444 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.334802 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:25.335385 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.335402 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.336743 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.336746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.337701 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.337712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.338458 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.338461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.338608 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.339214 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:25.339230 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:25.340663 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:25.342785 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.342790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.342800 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.343073 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.343455 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.343458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.343459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.343596 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.344010 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:25.344015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:25.344237 (+ 222us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.344394 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.344717 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":150,"threads_started":1}
I20260810 22:42:25.346953 5613 heartbeater.cc:347] Connected to a master server at 127.19.202.126:37125
I20260810 22:42:25.347230 5613 heartbeater.cc:464] Registering TS with master...
I20260810 22:42:25.347779 5613 heartbeater.cc:511] Master 127.19.202.126:37125 requested a full tablet report, sending...
I20260810 22:42:25.348961 5250 ts_manager.cc:194] Registered new tserver with Master: 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077)
I20260810 22:42:25.349529 5250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:33569
W20260810 22:42:25.445751 5616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:25.445999 5616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:25.446071 5616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:25.450066 5616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:25.450121 5616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:25.450230 5616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:42:25.455353 5616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:44741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=39913
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:25.456388 5616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:25.457489 5616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:25.466984 5625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:25.466969 5623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:25.466971 5622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:25.467260 5616 server_base.cc:1034] running on GCE node
I20260810 22:42:25.467695 5616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:25.468345 5616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:25.469516 5616 hybrid_clock.cc:648] HybridClock initialized: now 1786401745469490 us; error 42 us; skew 500 ppm
Aug 10 22:42:25 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401745, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:25.472152 5616 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:42:25.473064 5616 webserver.cc:460] Webserver started at http://127.19.202.67:39913/ using document root <none> and password file <none>
I20260810 22:42:25.473680 5616 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:25.473788 5616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:25.477849 5616 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:42:25.479894 5634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:25.481273 5616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:42:25.481384 5616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "5eee40525a8f4af48f2568dbbe6f014e"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a620bc255d70c3642229d60ddaa256a"
server_key_iv: "357857e7546c8c7d127e6f94c4fd2bb3"
server_key_version: "encryptionkey@0"
I20260810 22:42:25.481886 5616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:25.492193 5616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:25.494948 5616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:25.495126 5616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:25.495674 5616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:25.496452 5616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:42:25.496497 5616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:25.496559 5616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:42:25.496611 5616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:25.505965 5616 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:44741
I20260810 22:42:25.505985 5762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:44741 every 8 connection(s)
I20260810 22:42:25.506939 5616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:42:25.507710 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 5616
Aug 10 22:42:25 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401745, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:42:25.517769 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.508437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.19.202.126:37125)
0810 22:42:25.508591 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.508596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.508839 (+ 243us) server_negotiation.cc:407] Connection header received
0810 22:42:25.509009 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.509014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.509075 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.509166 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:25.509976 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.510611 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.511784 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.512213 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.512338 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.514213 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:25.514216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:25.514218 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:25.514240 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:25.515854 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.516367 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.516369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.516371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.516448 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.516857 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.516866 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.516868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.517128 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:25.517364 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.517481 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.517615 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:42:25.521713 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.508962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:33333 (local address 127.19.202.126:37125)
0810 22:42:25.509169 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.509173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.509899 (+ 726us) server_negotiation.cc:407] Connection header received
0810 22:42:25.511243 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.511246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.511290 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.511374 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:25.512992 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.513491 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.514838 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.515200 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.515313 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.518172 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:25.518176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:25.518182 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:25.518211 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:25.519845 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.520331 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.520334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.520336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.520425 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.520724 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.520726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.520727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.520939 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:25.521023 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.521489 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.521593 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
I20260810 22:42:25.522527 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.509269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:33333)
0810 22:42:25.509717 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.509777 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.511065 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.511558 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.511570 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.512102 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:25.512845 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.512871 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.513605 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.513609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.514723 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.514733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.515332 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.515335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.515499 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.516044 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:25.516068 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:25.517990 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:25.519953 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.519958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.519969 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.520220 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.520503 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:25.520505 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:25.520506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:25.520644 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:25.521032 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:25.521037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:25.521252 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.521415 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.521669 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":139,"threads_started":1}
I20260810 22:42:25.523723 5763 heartbeater.cc:347] Connected to a master server at 127.19.202.126:37125
I20260810 22:42:25.523973 5763 heartbeater.cc:464] Registering TS with master...
I20260810 22:42:25.524484 5763 heartbeater.cc:511] Master 127.19.202.126:37125 requested a full tablet report, sending...
I20260810 22:42:25.525516 5250 ts_manager.cc:194] Registered new tserver with Master: 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
I20260810 22:42:25.526063 5250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:33333
I20260810 22:42:25.531360 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:42:25.535094 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.531944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52992 (local address 127.19.202.126:37125)
0810 22:42:25.532060 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.532063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.532249 (+ 186us) server_negotiation.cc:407] Connection header received
0810 22:42:25.532364 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.532366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.532457 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.532536 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:42:25.532733 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.533284 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.534077 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.534286 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.534373 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.534683 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:42:25.534802 (+ 119us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 22:42:25.535004 (+ 202us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52992: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 22:42:25.535171 5766 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52992: token expired
W20260810 22:42:25.535526 5168 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.19.202.126:37125)
I20260810 22:42:25.542460 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.535851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.19.202.126:37125)
0810 22:42:25.535946 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.535950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.535968 (+ 18us) server_negotiation.cc:407] Connection header received
0810 22:42:25.536085 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.536087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.536120 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.536184 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:25.537092 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.537547 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.538285 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.538492 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.538568 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.539786 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:25.539788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:25.539790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:25.539811 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:25.541147 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.541539 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.541542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.541543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.541615 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:25.541828 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:25.541830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:25.541831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:25.541979 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:25.542077 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.542257 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.542348 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 22:42:25.544727 5250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:42:25.547902 5250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:42:25.555447 5384 tablet_service.cc:1467] Processing CreateTablet for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c (DEFAULT_TABLE table=test-table [id=c6eaa73b8992434b92a0370f38908e45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:42:25.556135 5384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:42:25.559106 5339 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Bootstrap starting.
I20260810 22:42:25.559692 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.554293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:35077 (local address 127.0.0.1:42666)
0810 22:42:25.554454 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.554469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.554538 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.555433 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.555436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.555452 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:25.555547 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.555555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.556924 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.556927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.558189 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.558199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.559265 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.559268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.559365 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.559409 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.559475 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.559553 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":43}
I20260810 22:42:25.559844 5774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.554831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:44741 (local address 127.0.0.1:51068)
0810 22:42:25.555182 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.555196 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.555283 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.556112 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.556115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.556132 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:25.556318 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.556326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.557438 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.557443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.558591 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.558598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.559487 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.559490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.559581 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.559600 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.559657 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.559719 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260810 22:42:25.559939 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.555248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.19.202.67:44741)
0810 22:42:25.555722 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.555735 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.555755 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:42:25.555829 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.555834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.555982 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.556135 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:25.556443 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.557329 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.558697 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.559358 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.559484 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.559630 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.559686 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.559770 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":132,"threads_started":1}
I20260810 22:42:25.559988 5339 tablet_bootstrap.cc:654] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Neither blocks nor log segments found. Creating new log.
I20260810 22:42:25.560582 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.554554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.19.202.66:35077)
0810 22:42:25.554956 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.554973 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.554997 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:42:25.555079 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.555085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.555293 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.555469 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:25.555668 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.556766 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.558311 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.559135 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.559939 (+ 804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.560105 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.560146 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.560254 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":161,"threads_started":1}
I20260810 22:42:25.561513 5339 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: No bootstrap required, opened a new log
I20260810 22:42:25.561601 5339 ts_tablet_manager.cc:1397] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:42:25.562041 5339 raft_consensus.cc:348] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.562163 5339 raft_consensus.cc:374] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:42:25.562201 5339 raft_consensus.cc:729] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662782d9a7cf46558b4f73392916c5e8, State: Initialized, Role: FOLLOWER
I20260810 22:42:25.562331 5339 consensus_queue.cc:260] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.562609 5339 ts_tablet_manager.cc:1428] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:42:25.562906 5685 tablet_service.cc:1467] Processing CreateTablet for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c (DEFAULT_TABLE table=test-table [id=c6eaa73b8992434b92a0370f38908e45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:42:25.563306 5535 tablet_service.cc:1467] Processing CreateTablet for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c (DEFAULT_TABLE table=test-table [id=c6eaa73b8992434b92a0370f38908e45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:42:25.563923 5685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:42:25.564103 5535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:42:25.570755 5777 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Bootstrap starting.
I20260810 22:42:25.570779 5778 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Bootstrap starting.
I20260810 22:42:25.573247 5778 tablet_bootstrap.cc:654] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Neither blocks nor log segments found. Creating new log.
I20260810 22:42:25.573454 5777 tablet_bootstrap.cc:654] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Neither blocks nor log segments found. Creating new log.
I20260810 22:42:25.574143 5778 log.cc:824] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:25.574280 5777 log.cc:824] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:25.576038 5778 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: No bootstrap required, opened a new log
I20260810 22:42:25.576051 5777 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: No bootstrap required, opened a new log
I20260810 22:42:25.576238 5778 ts_tablet_manager.cc:1397] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 22:42:25.576261 5777 ts_tablet_manager.cc:1397] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:42:25.579129 5777 raft_consensus.cc:348] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.579129 5778 raft_consensus.cc:348] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.579342 5777 raft_consensus.cc:374] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:42:25.579368 5778 raft_consensus.cc:374] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:42:25.579399 5777 raft_consensus.cc:729] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eee40525a8f4af48f2568dbbe6f014e, State: Initialized, Role: FOLLOWER
I20260810 22:42:25.579449 5778 raft_consensus.cc:729] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a1b3b45fa74f888f04b51b12aa548d, State: Initialized, Role: FOLLOWER
I20260810 22:42:25.579875 5777 consensus_queue.cc:260] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.579878 5778 consensus_queue.cc:260] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.580478 5613 heartbeater.cc:503] Master 127.19.202.126:37125 was elected leader, sending a full tablet report...
I20260810 22:42:25.580478 5763 heartbeater.cc:503] Master 127.19.202.126:37125 was elected leader, sending a full tablet report...
I20260810 22:42:25.580852 5778 ts_tablet_manager.cc:1428] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260810 22:42:25.580852 5777 ts_tablet_manager.cc:1428] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260810 22:42:25.642633 5458 tablet.cc:2369] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:42:25.757897 5764 tablet.cc:2369] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:42:25.767650 5781 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:25.767853 5781 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.769201 5781 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741), 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005)
I20260810 22:42:25.773741 5615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.769514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35005 (local address 127.19.202.66:41883)
0810 22:42:25.769654 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.769671 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.769780 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.770143 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.770146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.770164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:25.770282 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.770288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.771041 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.771044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.772055 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.772065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.773384 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.773387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.773473 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.773494 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.773544 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.773597 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 22:42:25.773905 5783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.769605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:41883 (local address 127.19.202.65:35005)
0810 22:42:25.769886 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.769890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.769904 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:42:25.769954 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.769956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.770010 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.770130 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:25.770406 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.770926 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.772232 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.773263 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.773404 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.773581 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.773627 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.773718 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":122,"threads_started":1}
I20260810 22:42:25.774245 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.769708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:58853 (local address 127.19.202.67:44741)
0810 22:42:25.769845 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.769853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.770352 (+ 499us) server_negotiation.cc:407] Connection header received
0810 22:42:25.770506 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.770509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.770560 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.770639 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:42:25.770951 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.771431 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.773012 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.773597 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.773732 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.773785 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.774024 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.774121 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:42:25.774636 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.769525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:44741 (local address 127.19.202.66:58853)
0810 22:42:25.770245 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.770257 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:25.770393 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:25.770662 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:25.770665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:25.770683 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:42:25.770824 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.770830 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.771588 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.771591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.772884 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:25.772894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.773730 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.773733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:25.773832 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.773863 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 22:42:25.774405 (+ 542us) client_negotiation.cc:240] Negotiation successful
0810 22:42:25.774479 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":370,"thread_start_us":137,"threads_started":1}
I20260810 22:42:25.774544 5404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f653db4b8ccd4b3cb10c0afd5e57ab3c" candidate_uuid: "77a1b3b45fa74f888f04b51b12aa548d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662782d9a7cf46558b4f73392916c5e8" is_pre_election: true
I20260810 22:42:25.774919 5404 raft_consensus.cc:2454] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77a1b3b45fa74f888f04b51b12aa548d in term 0.
I20260810 22:42:25.775255 5708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f653db4b8ccd4b3cb10c0afd5e57ab3c" candidate_uuid: "77a1b3b45fa74f888f04b51b12aa548d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eee40525a8f4af48f2568dbbe6f014e" is_pre_election: true
I20260810 22:42:25.775439 5487 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8, 77a1b3b45fa74f888f04b51b12aa548d; no voters:
I20260810 22:42:25.775544 5708 raft_consensus.cc:2454] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77a1b3b45fa74f888f04b51b12aa548d in term 0.
I20260810 22:42:25.775774 5781 raft_consensus.cc:2781] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:42:25.775863 5781 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:42:25.775928 5781 raft_consensus.cc:3037] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:42:25.777079 5781 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.777419 5781 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [CANDIDATE]: Term 1 election: Requested vote from peers 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741), 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005)
I20260810 22:42:25.777791 5708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f653db4b8ccd4b3cb10c0afd5e57ab3c" candidate_uuid: "77a1b3b45fa74f888f04b51b12aa548d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eee40525a8f4af48f2568dbbe6f014e"
I20260810 22:42:25.777892 5404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f653db4b8ccd4b3cb10c0afd5e57ab3c" candidate_uuid: "77a1b3b45fa74f888f04b51b12aa548d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662782d9a7cf46558b4f73392916c5e8"
I20260810 22:42:25.777951 5708 raft_consensus.cc:3037] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:42:25.778051 5404 raft_consensus.cc:3037] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:42:25.779027 5708 raft_consensus.cc:2454] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77a1b3b45fa74f888f04b51b12aa548d in term 1.
I20260810 22:42:25.779036 5404 raft_consensus.cc:2454] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77a1b3b45fa74f888f04b51b12aa548d in term 1.
I20260810 22:42:25.779304 5487 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8, 77a1b3b45fa74f888f04b51b12aa548d; no voters:
I20260810 22:42:25.779526 5781 raft_consensus.cc:2781] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:42:25.779752 5781 raft_consensus.cc:686] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 1 LEADER]: Becoming Leader. State: Replica: 77a1b3b45fa74f888f04b51b12aa548d, State: Running, Role: LEADER
I20260810 22:42:25.780068 5781 consensus_queue.cc:237] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:25.783247 5248 catalog_manager.cc:5495] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d reported cstate change: term changed from 0 to 1, leader changed from <none> to 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66). New cstate: current_term: 1 leader_uuid: "77a1b3b45fa74f888f04b51b12aa548d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } health_report { overall_health: UNKNOWN } } }
I20260810 22:42:25.800503 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:25.797065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.19.202.66:35077)
0810 22:42:25.797212 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:25.797217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:25.797230 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:42:25.797386 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:25.797389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:25.797452 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:25.797556 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:42:25.797792 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.798298 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.799098 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:25.799322 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:25.799424 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:25.799728 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:42:25.800110 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:42:25.800234 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:25.800294 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 22:42:25.800348 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:42:25.806527 5708 raft_consensus.cc:1261] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Refusing update from remote peer 77a1b3b45fa74f888f04b51b12aa548d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:42:25.806522 5404 raft_consensus.cc:1261] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Refusing update from remote peer 77a1b3b45fa74f888f04b51b12aa548d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:42:25.807279 5785 consensus_queue.cc:1035] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:42:25.807472 5781 consensus_queue.cc:1035] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [LEADER]: Connected to new peer: Peer: permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:42:25.813268 5788 mvcc.cc:205] Tried to move back new op lower bound from 7317101550815354880 to 7317101550717878272. Current Snapshot: MvccSnapshot[applied={T|T < 7317101550815354880}]
I20260810 22:42:25.813375 5789 mvcc.cc:205] Tried to move back new op lower bound from 7317101550815354880 to 7317101550717878272. Current Snapshot: MvccSnapshot[applied={T|T < 7317101550815354880}]
I20260810 22:42:25.824112 5248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:42:25.824280 5248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:42:25.826519 5248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Sending DeleteTablet for 3 replicas of tablet f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:25.827186 5535 tablet_service.cc:1514] Processing DeleteTablet for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:42:25 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:42666
I20260810 22:42:25.827282 5685 tablet_service.cc:1514] Processing DeleteTablet for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:42:25 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:51068
I20260810 22:42:25.827445 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 5290
I20260810 22:42:25.830389 5800 tablet_replica.cc:330] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: stopping tablet replica
I20260810 22:42:25.831041 5800 raft_consensus.cc:2229] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:42:25.831555 5800 raft_consensus.cc:2258] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:42:25.833653 5800 ts_tablet_manager.cc:1905] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:42:25.836686 5800 ts_tablet_manager.cc:1918] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:42:25.836826 5800 log.cc:1196] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:25.837131 5801 tablet_replica.cc:330] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: stopping tablet replica
I20260810 22:42:25.837210 5800 ts_tablet_manager.cc:1939] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 77a1b3b45fa74f888f04b51b12aa548d: Deleting consensus metadata
I20260810 22:42:25.837596 5801 raft_consensus.cc:2229] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:42:25.837994 5801 raft_consensus.cc:2258] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:42:25.838477 5234 catalog_manager.cc:4841] TS 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): tablet f653db4b8ccd4b3cb10c0afd5e57ab3c (table test-table [id=c6eaa73b8992434b92a0370f38908e45]) successfully deleted
W20260810 22:42:25.839083 5235 connection.cc:535] client connection to 127.19.202.65:35005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 22:42:25.839247 5235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 22:42:25.839762 5801 ts_tablet_manager.cc:1905] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 22:42:25.840473 5774 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:42:25.840148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35005 (local address 127.0.0.1:54448)
0810 22:42:25.840297 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:25.840384 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35005: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 22:42:25.840615 5235 catalog_manager.cc:4584] TS 662782d9a7cf46558b4f73392916c5e8 (127.19.202.65:35005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f653db4b8ccd4b3cb10c0afd5e57ab3c: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35005: connect: Connection reset by peer (error 104)
I20260810 22:42:25.841495 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 5461
I20260810 22:42:25.844144 5801 ts_tablet_manager.cc:1918] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:42:25.844246 5801 log.cc:1196] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:25.844606 5801 ts_tablet_manager.cc:1939] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 5eee40525a8f4af48f2568dbbe6f014e: Deleting consensus metadata
I20260810 22:42:25.845973 5233 catalog_manager.cc:4841] TS 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): tablet f653db4b8ccd4b3cb10c0afd5e57ab3c (table test-table [id=c6eaa73b8992434b92a0370f38908e45]) successfully deleted
I20260810 22:42:25.850409 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 5616
I20260810 22:42:25.858592 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 5214
I20260810 22:42:45.867758 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37125
--webserver_interface=127.19.202.126
--webserver_port=36175
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:42:45.983186 5803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:45.983460 5803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:45.983500 5803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:45.987358 5803 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 22:42:45.987411 5803 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 22:42:45.987426 5803 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 22:42:45.987452 5803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:42:45.987474 5803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:45.987531 5803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:45.987556 5803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:42:45.987583 5803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:42:45.992801 5803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=36175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:45.993906 5803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:45.994956 5803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:46.004088 5813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:46.004088 5810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:46.004230 5809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:46.004261 5803 server_base.cc:1034] running on GCE node
I20260810 22:42:46.005108 5803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:46.006196 5803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:46.007380 5803 hybrid_clock.cc:648] HybridClock initialized: now 1786401766007367 us; error 24 us; skew 500 ppm
Aug 10 22:42:46 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401766, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:46.010346 5803 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:42:46.011286 5803 webserver.cc:460] Webserver started at http://127.19.202.126:36175/ using document root <none> and password file <none>
I20260810 22:42:46.011909 5803 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:46.011994 5803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:46.015947 5803 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:42:46.018172 5820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.019364 5803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:42:46.019511 5803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "e8593e37e2dd419299520249db4b89da"
format_stamp: "Formatted at 2026-08-10 22:42:03 on dist-test-slave-1x4g"
server_key: "6d1c9467486ee970c2899d6d60b02b62"
server_key_iv: "c874adf321bcd5099179811704690dcc"
server_key_version: "encryptionkey@0"
I20260810 22:42:46.020084 5803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:46.044135 5803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:46.047385 5803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:46.047580 5803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:46.055143 5803 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:37125
I20260810 22:42:46.055416 5884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:37125 every 8 connection(s)
I20260810 22:42:46.056246 5803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:42:46.060598 5887 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:42:46.062237 5887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Bootstrap starting.
I20260810 22:42:46.066033 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 5803
I20260810 22:42:46.066481 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:35005
--local_ip_for_outbound_sockets=127.19.202.65
--tserver_master_addrs=127.19.202.126:37125
--webserver_port=41875
--webserver_interface=127.19.202.65
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:42:46.080677 5887 log.cc:824] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:46.091864 5887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:42:46.092243 5887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Bootstrap complete.
I20260810 22:42:46.096343 5887 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:46.096820 5887 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8593e37e2dd419299520249db4b89da, State: Initialized, Role: FOLLOWER
I20260810 22:42:46.097323 5887 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:46.097441 5887 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:42:46.097525 5887 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:42:46.097638 5887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 2 FOLLOWER]: Advancing to term 3
I20260810 22:42:46.099495 5887 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:46.099824 5887 leader_election.cc:302] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8593e37e2dd419299520249db4b89da; no voters:
I20260810 22:42:46.100162 5887 leader_election.cc:288] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 22:42:46.100291 5891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 3 FOLLOWER]: Leader election won for term 3
I20260810 22:42:46.100564 5891 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [term 3 LEADER]: Becoming Leader. State: Replica: e8593e37e2dd419299520249db4b89da, State: Running, Role: LEADER
I20260810 22:42:46.100883 5891 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } }
I20260810 22:42:46.101470 5887 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:42:46.102344 5892 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e8593e37e2dd419299520249db4b89da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } } }
I20260810 22:42:46.102483 5892 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: This master's current role is: LEADER
I20260810 22:42:46.103165 5893 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8593e37e2dd419299520249db4b89da. Latest consensus state: current_term: 3 leader_uuid: "e8593e37e2dd419299520249db4b89da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8593e37e2dd419299520249db4b89da" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37125 } } }
I20260810 22:42:46.103259 5893 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da [sys.catalog]: This master's current role is: LEADER
I20260810 22:42:46.103399 5901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:42:46.108314 5901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:42:46.108991 5901 catalog_manager.cc:1259] Loaded cluster ID: 595527617bda4cc888db28562993bf3c
I20260810 22:42:46.109051 5901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:42:46.111605 5901 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:42:46.112406 5901 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Loaded TSK: 0
I20260810 22:42:46.112498 5901 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8593e37e2dd419299520249db4b89da: Loaded TSK: 1
F20260810 22:42:46.113263 5901 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe94c35b9ad google::LogMessage::Fail() at ??:0
@ 0x7fe94c35e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe94c35b4de google::LogMessage::Flush() at ??:0
@ 0x7fe94c35ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe94f01f18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe94f017f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe956cc07bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe956c9a99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe956c9b178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fe956cdcdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe956cfb82b std::function<>::operator()() at ??:0
@ 0x7fe956c9ab4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe956c9bf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe956c98579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe956cdc640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe94e5ad3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe94e5adc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe94e5af3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe94e59e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe94fbe26db start_thread at ??:0
@ 0x7fe94ae5961f clone at ??:0
W20260810 22:42:46.203634 5889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:46.203861 5889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:46.203908 5889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:46.208357 5889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:46.208436 5889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:46.208508 5889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:42:46.215714 5889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:35005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=41875
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:46.216892 5889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:46.217902 5889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:46.227787 5939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:46.227795 5940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:46.228022 5944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:46.228591 5889 server_base.cc:1034] running on GCE node
I20260810 22:42:46.229058 5889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:46.229691 5889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:46.230876 5889 hybrid_clock.cc:648] HybridClock initialized: now 1786401766230838 us; error 48 us; skew 500 ppm
Aug 10 22:42:46 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401766, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:46.233757 5889 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:42:46.234751 5889 webserver.cc:460] Webserver started at http://127.19.202.65:41875/ using document root <none> and password file <none>
I20260810 22:42:46.235349 5889 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:46.235456 5889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:46.239586 5889 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260810 22:42:46.241950 5951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.243259 5889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:42:46.243386 5889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "662782d9a7cf46558b4f73392916c5e8"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "95263a6f4e3d7697cb27aaca6949262b"
server_key_iv: "910067664ec431c51b6612ae5ba803de"
server_key_version: "encryptionkey@0"
I20260810 22:42:46.243956 5889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:46.269008 5889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:46.272379 5889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:46.272585 5889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:46.273283 5889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:46.274441 5959 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 22:42:46.280084 5889 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 22:42:46.280184 5889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260810 22:42:46.280287 5889 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 22:42:46.282109 5889 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 22:42:46.282176 5889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260810 22:42:46.282491 5959 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Bootstrap starting.
I20260810 22:42:46.293548 5889 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:35005
I20260810 22:42:46.293808 6079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:35005 every 8 connection(s)
I20260810 22:42:46.294607 5889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:42:46.295593 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 5889
I20260810 22:42:46.296036 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:35077
--local_ip_for_outbound_sockets=127.19.202.66
--tserver_master_addrs=127.19.202.126:37125
--webserver_port=41359
--webserver_interface=127.19.202.66
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:42:46.304872 5959 log.cc:824] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Log is configured to *not* fsync() on all Append() calls
I20260810 22:42:46.311759 5959 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:42:46.312140 5959 tablet_bootstrap.cc:492] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Bootstrap complete.
I20260810 22:42:46.313598 5959 ts_tablet_manager.cc:1397] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.016s
I20260810 22:42:46.316053 5959 raft_consensus.cc:348] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:46.316502 5959 raft_consensus.cc:729] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662782d9a7cf46558b4f73392916c5e8, State: Initialized, Role: FOLLOWER
I20260810 22:42:46.317058 5959 consensus_queue.cc:260] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:46.318014 5959 ts_tablet_manager.cc:1428] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
W20260810 22:42:46.386086 6084 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:42:46.297975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:60279)
0810 22:42:46.299196 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.299256 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:46.300320 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:46.385092 (+ 84772us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":984,"thread_start_us":154,"threads_started":1}
W20260810 22:42:46.386659 6080 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:37125 (0 consecutive failures): Network error: Failed to ping master at 127.19.202.126:37125: Client connection negotiation failed: client connection to 127.19.202.126:37125: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 22:42:46.387457 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.387143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:36955)
0810 22:42:46.387287 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.387371 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 22:42:46.388155 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.387933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:47033)
0810 22:42:46.388043 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.388099 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 22:42:46.388355 6080 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:42:46.439525 6083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:46.439765 6083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:46.439807 6083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:46.443527 6083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:46.443574 6083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:46.443624 6083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:42:46.448477 6083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:35077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=41359
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:46.449421 6083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:46.450344 6083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:46.459894 6094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:46.459894 6092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:46.459964 6091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:46.460157 6083 server_base.cc:1034] running on GCE node
I20260810 22:42:46.460588 6083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:46.461268 6083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:46.462455 6083 hybrid_clock.cc:648] HybridClock initialized: now 1786401766462438 us; error 27 us; skew 500 ppm
Aug 10 22:42:46 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401766, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:46.465368 6083 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:42:46.466261 6083 webserver.cc:460] Webserver started at http://127.19.202.66:41359/ using document root <none> and password file <none>
I20260810 22:42:46.466815 6083 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:46.466930 6083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:46.470857 6083 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:42:46.472961 6101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.474229 6083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:42:46.474336 6083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "77a1b3b45fa74f888f04b51b12aa548d"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a2cf561d82ae9742960df0a18c8127f"
server_key_iv: "af024af9f91f4c35181fe4aa1532467b"
server_key_version: "encryptionkey@0"
I20260810 22:42:46.474858 6083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:46.503605 6083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:46.506928 6083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:46.507133 6083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:46.507768 6083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:46.508662 6083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:42:46.508710 6083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.508750 6083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:42:46.508792 6083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.518415 6083 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:35077
I20260810 22:42:46.518443 6228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:35077 every 8 connection(s)
I20260810 22:42:46.519394 6083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:42:46.523285 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.521501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:46223)
0810 22:42:46.521996 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.522111 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":123,"threads_started":1}
W20260810 22:42:46.523741 6229 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:37125 (0 consecutive failures): Network error: Failed to ping master at 127.19.202.126:37125: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
I20260810 22:42:46.524343 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.524095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:33981)
0810 22:42:46.524193 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.524240 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 22:42:46.525035 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.524853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:48851)
0810 22:42:46.524947 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.524989 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:42:46.525238 6229 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 22:42:46.527999 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 6083
I20260810 22:42:46.528475 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:44741
--local_ip_for_outbound_sockets=127.19.202.67
--tserver_master_addrs=127.19.202.126:37125
--webserver_port=39913
--webserver_interface=127.19.202.67
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:42:46.641845 6232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:42:46.642163 6232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:42:46.642243 6232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:42:46.646255 6232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:42:46.646327 6232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:42:46.646410 6232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:42:46.651495 6232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:44741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=39913
--tserver_master_addrs=127.19.202.126:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:42:46.652570 6232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:42:46.653595 6232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:42:46.663396 6238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:42:46.663424 6237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:46.663473 6232 server_base.cc:1034] running on GCE node
W20260810 22:42:46.663396 6240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:42:46.664083 6232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:42:46.664767 6232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:42:46.665984 6232 hybrid_clock.cc:648] HybridClock initialized: now 1786401766665969 us; error 29 us; skew 500 ppm
Aug 10 22:42:46 dist-test-slave-1x4g krb5kdc[4612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401766, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:42:46.668742 6232 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:42:46.669623 6232 webserver.cc:460] Webserver started at http://127.19.202.67:39913/ using document root <none> and password file <none>
I20260810 22:42:46.670194 6232 fs_manager.cc:359] Metadata directory not provided
I20260810 22:42:46.670256 6232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:42:46.674050 6232 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:42:46.676059 6247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.677306 6232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:42:46.677425 6232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "5eee40525a8f4af48f2568dbbe6f014e"
format_stamp: "Formatted at 2026-08-10 22:42:04 on dist-test-slave-1x4g"
server_key: "0a620bc255d70c3642229d60ddaa256a"
server_key_iv: "357857e7546c8c7d127e6f94c4fd2bb3"
server_key_version: "encryptionkey@0"
I20260810 22:42:46.678062 6232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:42:46.707690 6232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:42:46.711257 6232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:42:46.711477 6232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:42:46.712226 6232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:42:46.713243 6232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:42:46.713310 6232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.713382 6232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:42:46.713421 6232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:42:46.722968 6232 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:44741
I20260810 22:42:46.722994 6370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:44741 every 8 connection(s)
I20260810 22:42:46.724031 6232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:42:46.725854 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 6232
I20260810 22:42:46.728344 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.726410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:59577)
0810 22:42:46.726986 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.727127 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":174,"threads_started":1}
W20260810 22:42:46.728914 6371 heartbeater.cc:644] Failed to heartbeat to 127.19.202.126:37125 (0 consecutive failures): Network error: Failed to ping master at 127.19.202.126:37125: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
I20260810 22:42:46.729538 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.729322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:56107)
0810 22:42:46.729434 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.729482 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 22:42:46.730074 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:46.729903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:35543)
0810 22:42:46.729990 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:46.730029 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 22:42:46.730242 6371 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 22:42:47.389394 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:47.388957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:58345)
0810 22:42:47.389226 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:47.389278 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260810 22:42:47.475507 6377 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:47.475694 6377 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:47.476924 6377 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
Aug 10 22:42:47 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401766, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.67@KRBTEST.COM
Aug 10 22:42:47 dist-test-slave-1x4g krb5kdc[4612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401766, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.66@KRBTEST.COM
I20260810 22:42:47.496361 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:47.477070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:35077 (local address 127.19.202.65:35079)
0810 22:42:47.477254 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:47.477268 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:47.477388 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:47.479543 (+ 2155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:47.479547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:47.479913 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:47.480384 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:47.480397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.481974 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.481979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:47.483602 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:47.483614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.484303 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.484306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:47.484459 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:47.489281 (+ 4822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:47.489298 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:47.491289 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:47.493466 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:47.493469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:47.493470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:47.493826 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:47.494241 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:47.494243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:47.494244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:47.494390 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:47.495145 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:47.495153 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:47.495532 (+ 379us) client_negotiation.cc:769] Sending connection context
0810 22:42:47.495758 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 22:42:47.496206 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":354,"spinlock_wait_cycles":2176}
I20260810 22:42:47.496383 6380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:47.477490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:55225 (local address 127.19.202.67:44741)
0810 22:42:47.477811 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:47.477822 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:47.477855 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:42:47.478674 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:47.478682 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:47.479137 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:47.479479 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:47.480557 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.481841 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.484447 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.484944 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.485103 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:47.491236 (+ 6133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:47.491244 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:47.491261 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:47.491295 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:47.493200 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:47.493930 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:47.493936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:47.493941 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:47.494086 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:47.494628 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:47.494632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:47.494634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:47.495060 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:47.495160 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:47.495931 (+ 771us) server_negotiation.cc:299] Negotiation successful
0810 22:42:47.496202 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
I20260810 22:42:47.496369 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:47.477150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:44741 (local address 127.19.202.65:55225)
0810 22:42:47.477465 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:47.477479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:42:47.477585 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:42:47.479976 (+ 2391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:42:47.479978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:42:47.480140 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:42:47.480382 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:47.480397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.482014 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.482016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:47.483602 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:42:47.483614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.485068 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.485071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:42:47.485154 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:47.489298 (+ 4144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:42:47.489311 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:42:47.490966 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:42:47.493323 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:47.493330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:47.493344 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:47.493793 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:47.494484 (+ 691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:42:47.494485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:42:47.494487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:42:47.494543 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:42:47.495162 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:42:47.495165 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:42:47.495532 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 22:42:47.495758 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 22:42:47.496152 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":262,"spinlock_wait_cycles":15744,"thread_start_us":90,"threads_started":1}
I20260810 22:42:47.496383 6379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:47.477255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:35079 (local address 127.19.202.66:35077)
0810 22:42:47.477664 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 22:42:47.477682 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:42:47.477729 (+ 47us) server_negotiation.cc:407] Connection header received
0810 22:42:47.478645 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:42:47.478654 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:42:47.479137 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:42:47.479479 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:42:47.480562 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.481834 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.483714 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:42:47.484197 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:42:47.484315 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:42:47.491479 (+ 7164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:42:47.491484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:42:47.491495 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:42:47.491522 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:42:47.493353 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:47.493958 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:47.493963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:47.493967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:47.494085 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:42:47.494476 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:42:47.494478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:42:47.494480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:42:47.495011 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:42:47.495160 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 22:42:47.495913 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 22:42:47.496202 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":151,"threads_started":1}
W20260810 22:42:47.499181 5953 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
W20260810 22:42:47.499373 5954 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:47.499454 5954 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8; no voters: 5eee40525a8f4af48f2568dbbe6f014e, 77a1b3b45fa74f888f04b51b12aa548d
I20260810 22:42:47.499703 6377 raft_consensus.cc:2726] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:42:47.526232 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:47.525766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:57615)
0810 22:42:47.526059 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:47.526119 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":74,"threads_started":1}
I20260810 22:42:47.731127 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:47.730729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:40193)
0810 22:42:47.730947 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:47.731000 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
I20260810 22:42:48.390529 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:48.390134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:57731)
0810 22:42:48.390371 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:48.390431 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260810 22:42:48.527375 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:48.526931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:56079)
0810 22:42:48.527201 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:48.527259 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260810 22:42:48.732240 6385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:48.731843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:54555)
0810 22:42:48.732082 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:48.732133 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":81,"threads_started":1}
I20260810 22:42:49.303514 6386 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:49.303635 6386 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:49.304029 6386 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
W20260810 22:42:49.304816 5953 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
W20260810 22:42:49.304943 5954 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:49.305027 5954 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8; no voters: 5eee40525a8f4af48f2568dbbe6f014e, 77a1b3b45fa74f888f04b51b12aa548d
I20260810 22:42:49.305167 6386 raft_consensus.cc:2726] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:42:49.391758 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:49.391326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:58401)
0810 22:42:49.391606 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:49.391660 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":108,"threads_started":1}
I20260810 22:42:49.528542 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:49.528126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:42609)
0810 22:42:49.528386 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:49.528447 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":94,"threads_started":1}
I20260810 22:42:49.733315 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:49.732932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:34743)
0810 22:42:49.733171 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:49.733223 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260810 22:42:50.393009 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:50.392540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:34215)
0810 22:42:50.392845 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:50.392902 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
I20260810 22:42:50.529691 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:50.529275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:41457)
0810 22:42:50.529540 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:50.529599 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":105,"threads_started":1}
I20260810 22:42:50.734469 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:50.734045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:41387)
0810 22:42:50.734314 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:50.734373 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
I20260810 22:42:51.051290 6393 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:51.051427 6393 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:51.051788 6393 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
W20260810 22:42:51.052634 5953 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
W20260810 22:42:51.052747 5954 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:51.052878 5954 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8; no voters: 5eee40525a8f4af48f2568dbbe6f014e, 77a1b3b45fa74f888f04b51b12aa548d
I20260810 22:42:51.053016 6393 raft_consensus.cc:2726] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:42:51.394380 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:51.393900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:52585)
0810 22:42:51.394222 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:51.394280 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":94,"threads_started":1}
I20260810 22:42:51.530869 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:51.530481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:43805)
0810 22:42:51.530722 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:51.530774 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260810 22:42:51.735620 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:51.735212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:54181)
0810 22:42:51.735470 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:51.735526 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260810 22:42:52.395498 6399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:52.395082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:41933)
0810 22:42:52.395344 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:52.395399 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260810 22:42:52.531985 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:52.531579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:38273)
0810 22:42:52.531828 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:52.531875 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":85,"threads_started":1}
I20260810 22:42:52.736744 6401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:52.736330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:59503)
0810 22:42:52.736557 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:52.736641 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
I20260810 22:42:53.011129 6402 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:53.011289 6402 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:53.011736 6402 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
W20260810 22:42:53.012527 5953 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
W20260810 22:42:53.012609 5954 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:53.012717 5954 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8; no voters: 5eee40525a8f4af48f2568dbbe6f014e, 77a1b3b45fa74f888f04b51b12aa548d
I20260810 22:42:53.012912 6402 raft_consensus.cc:2726] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:42:53.396920 6403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:53.396404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:48273)
0810 22:42:53.396699 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:53.396801 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":78,"threads_started":1}
I20260810 22:42:53.533226 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:53.532722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:46839)
0810 22:42:53.533082 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:53.533137 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":76,"threads_started":1}
I20260810 22:42:53.737982 6405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:53.737506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:58161)
0810 22:42:53.737823 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:53.737888 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":90,"threads_started":1}
I20260810 22:42:54.398120 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:54.397674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:40635)
0810 22:42:54.397968 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:54.398024 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":106,"threads_started":1}
I20260810 22:42:54.534507 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:54.534014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:34435)
0810 22:42:54.534346 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:54.534409 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":105,"threads_started":1}
I20260810 22:42:54.739145 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:54.738708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:58989)
0810 22:42:54.738994 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:54.739049 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":90,"threads_started":1}
I20260810 22:42:55.399353 6409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:55.398924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:57075)
0810 22:42:55.399198 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:55.399255 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
I20260810 22:42:55.535705 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:55.535286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:35629)
0810 22:42:55.535557 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:55.535612 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
I20260810 22:42:55.740406 6411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:55.739968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:51443)
0810 22:42:55.740254 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:55.740309 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":116,"threads_started":1}
I20260810 22:42:56.090339 6412 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:42:56.090494 6412 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:42:56.090971 6412 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
W20260810 22:42:56.091754 5953 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
W20260810 22:42:56.091842 5954 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:42:56.091955 5954 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8; no voters: 5eee40525a8f4af48f2568dbbe6f014e, 77a1b3b45fa74f888f04b51b12aa548d
I20260810 22:42:56.092119 6412 raft_consensus.cc:2726] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:42:56.400597 6413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:56.400164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:40127)
0810 22:42:56.400432 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:56.400486 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260810 22:42:56.536935 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:56.536503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:39325)
0810 22:42:56.536787 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:56.536840 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":104,"threads_started":1}
I20260810 22:42:56.741565 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:56.741170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:54749)
0810 22:42:56.741421 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:56.741473 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":88,"threads_started":1}
I20260810 22:42:57.401872 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:57.401453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:52595)
0810 22:42:57.401723 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:57.401775 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":84,"threads_started":1}
I20260810 22:42:57.538129 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:57.537698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:42063)
0810 22:42:57.537966 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:57.538035 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":94,"threads_started":1}
I20260810 22:42:57.742676 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:57.742290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:51067)
0810 22:42:57.742536 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:57.742588 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
I20260810 22:42:58.403076 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:58.402612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:40997)
0810 22:42:58.402916 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:58.402981 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":87,"threads_started":1}
I20260810 22:42:58.539232 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:58.538821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:55921)
0810 22:42:58.539070 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:58.539132 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":81,"threads_started":1}
I20260810 22:42:58.743769 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:58.743336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:43665)
0810 22:42:58.743605 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:58.743662 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260810 22:42:59.404428 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:59.403962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:60709)
0810 22:42:59.404257 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:59.404318 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":100,"threads_started":1}
I20260810 22:42:59.540381 6424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:59.539984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:34217)
0810 22:42:59.540233 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:59.540287 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
I20260810 22:42:59.744959 6425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:42:59.744538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:55203)
0810 22:42:59.744807 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 22:42:59.744862 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
I20260810 22:43:00.405679 6427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:00.405249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:51763)
0810 22:43:00.405524 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:00.405579 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":105,"threads_started":1}
I20260810 22:43:00.541630 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:00.541221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:33501)
0810 22:43:00.541461 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:00.541516 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":80,"threads_started":1}
I20260810 22:43:00.746095 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:00.745672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:43019)
0810 22:43:00.745942 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:00.745998 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260810 22:43:01.007154 6431 raft_consensus.cc:482] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:01.007288 6431 raft_consensus.cc:504] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a1b3b45fa74f888f04b51b12aa548d" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 35077 } } peers { permanent_uuid: "5eee40525a8f4af48f2568dbbe6f014e" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 44741 } } peers { permanent_uuid: "662782d9a7cf46558b4f73392916c5e8" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35005 } }
I20260810 22:43:01.007788 6431 leader_election.cc:288] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077), 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741)
W20260810 22:43:01.008586 5954 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77a1b3b45fa74f888f04b51b12aa548d (127.19.202.66:35077): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
W20260810 22:43:01.008685 5953 leader_election.cc:341] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5eee40525a8f4af48f2568dbbe6f014e (127.19.202.67:44741): Not found: Tablet not found: f653db4b8ccd4b3cb10c0afd5e57ab3c
I20260810 22:43:01.008849 5953 leader_election.cc:302] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 662782d9a7cf46558b4f73392916c5e8; no voters: 5eee40525a8f4af48f2568dbbe6f014e, 77a1b3b45fa74f888f04b51b12aa548d
I20260810 22:43:01.008989 6431 raft_consensus.cc:2726] T f653db4b8ccd4b3cb10c0afd5e57ab3c P 662782d9a7cf46558b4f73392916c5e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:43:01.407079 6432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:01.406635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.65:46919)
0810 22:43:01.406925 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:01.406983 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260810 22:43:01.542866 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:01.542466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.66:40817)
0810 22:43:01.542713 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:01.542768 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260810 22:43:01.733904 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 5889
I20260810 22:43:01.744184 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 6083
I20260810 22:43:01.747442 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:01.747025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:37125 (local address 127.19.202.67:59095)
0810 22:43:01.747292 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:01.747345 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.126:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
I20260810 22:43:01.751884 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 6232
2026-08-10T22:43:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59268 ms)
[----------] 30 tests from SecurityITest (244744 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:01 dist-test-slave-1x4g krb5kdc[6438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:01 dist-test-slave-1x4g krb5kdc[6438](info): set up 2 sockets
Aug 10 22:43:01 dist-test-slave-1x4g krb5kdc[6438](info): commencing operation
krb5kdc: starting...
W20260810 22:43:02.860026 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:02 dist-test-slave-1x4g krb5kdc[6438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:02.985973 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:46153
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:46153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:03.094333 6452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:03.094628 6452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:03.094700 6452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:03.098698 6452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:03.098753 6452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:03.098771 6452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:03.098788 6452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:03.098802 6452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:03.103833 6452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:46153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:46153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:03.104856 6452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:03.105805 6452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:43:03.114181 6452 server_base.cc:1034] running on GCE node
W20260810 22:43:03.114147 6457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:03.114171 6458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:03.114147 6460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:03.114923 6452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:03.115947 6452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:03.117110 6452 hybrid_clock.cc:648] HybridClock initialized: now 1786401783117089 us; error 30 us; skew 500 ppm
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:03.119856 6452 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:03.120707 6452 webserver.cc:460] Webserver started at http://127.19.202.126:39095/ using document root <none> and password file <none>
I20260810 22:43:03.121263 6452 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:03.121351 6452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:03.121577 6452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:03.123502 6452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "48ed5fd8df9342f99127dce1540d4c6c"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "1bcf83026ff766b6b9969fd12485464f"
server_key_iv: "4fefc41acdb720d2f6bd7095e95d4917"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.124011 6452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "48ed5fd8df9342f99127dce1540d4c6c"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "1bcf83026ff766b6b9969fd12485464f"
server_key_iv: "4fefc41acdb720d2f6bd7095e95d4917"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.127609 6452 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 22:43:03.129936 6470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.131148 6452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:03.131273 6452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "48ed5fd8df9342f99127dce1540d4c6c"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "1bcf83026ff766b6b9969fd12485464f"
server_key_iv: "4fefc41acdb720d2f6bd7095e95d4917"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.131402 6452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:03.153331 6452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:03.156555 6452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:03.156854 6452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:03.164002 6452 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:46153
I20260810 22:43:03.164031 6529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:46153 every 8 connection(s)
I20260810 22:43:03.164939 6452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:03.167503 6530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:03.172406 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 6452
I20260810 22:43:03.172549 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:03.172857 20265 external_mini_cluster.cc:1428] Setting key 31e5a92845dd4c9c93bcb5fb0eaf6c65
I20260810 22:43:03.173152 6530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c: Bootstrap starting.
I20260810 22:43:03.175657 6530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:03.176369 6530 log.cc:824] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:03.178314 6530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c: No bootstrap required, opened a new log
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:03.182014 6530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ed5fd8df9342f99127dce1540d4c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46153 } }
I20260810 22:43:03.182266 6530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:03.182349 6530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ed5fd8df9342f99127dce1540d4c6c, State: Initialized, Role: FOLLOWER
I20260810 22:43:03.182802 6530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ed5fd8df9342f99127dce1540d4c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46153 } }
I20260810 22:43:03.182955 6530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:03.183033 6530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:03.183141 6530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:03.183956 6530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ed5fd8df9342f99127dce1540d4c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46153 } }
I20260810 22:43:03.184316 6530 leader_election.cc:302] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48ed5fd8df9342f99127dce1540d4c6c; no voters:
I20260810 22:43:03.184666 6530 leader_election.cc:288] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:03.184752 6537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:03.185027 6537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [term 1 LEADER]: Becoming Leader. State: Replica: 48ed5fd8df9342f99127dce1540d4c6c, State: Running, Role: LEADER
I20260810 22:43:03.185398 6537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ed5fd8df9342f99127dce1540d4c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46153 } }
I20260810 22:43:03.185880 6530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:03.186643 6537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48ed5fd8df9342f99127dce1540d4c6c. Latest consensus state: current_term: 1 leader_uuid: "48ed5fd8df9342f99127dce1540d4c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ed5fd8df9342f99127dce1540d4c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46153 } } }
I20260810 22:43:03.187417 6537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:03.187304 6538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48ed5fd8df9342f99127dce1540d4c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ed5fd8df9342f99127dce1540d4c6c" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46153 } } }
I20260810 22:43:03.187765 6538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:03.188162 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.174234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.19.202.126:46153)
0810 22:43:03.174671 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.174685 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.174719 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:43:03.175322 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.175338 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.175651 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.175997 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:03.176866 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.177877 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.178779 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.179240 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.179413 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.182189 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:03.182194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:03.182206 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:03.182231 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:03.184114 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.184588 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.184592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.184597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.184791 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.185172 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.185175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.185176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.185683 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:03.185866 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.186761 (+ 895us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.187091 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":145,"threads_started":1}
I20260810 22:43:03.188635 6550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:03.191738 6550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:03.197255 6550 catalog_manager.cc:1347] Generated new cluster ID: 604ede1806a4434ea32f67d158e80132
I20260810 22:43:03.197319 6550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:03.218111 6550 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:03.219084 6550 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:03.223673 6550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c: Generated new TSK 0
I20260810 22:43:03.224387 6550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:43:03.261360 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:46153
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:03.378069 6560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:03.378346 6560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:03.378422 6560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:03.382501 6560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:03.382578 6560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:03.382658 6560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:43:03.387879 6560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:46153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:03.389014 6560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:03.390044 6560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:03.399448 6567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:03.399617 6560 server_base.cc:1034] running on GCE node
W20260810 22:43:03.399431 6569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:03.399464 6566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:03.400076 6560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:03.400630 6560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:03.401793 6560 hybrid_clock.cc:648] HybridClock initialized: now 1786401783401771 us; error 37 us; skew 500 ppm
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:03.404443 6560 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:43:03.405242 6560 webserver.cc:460] Webserver started at http://127.19.202.65:45125/ using document root <none> and password file <none>
I20260810 22:43:03.405712 6560 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:03.405766 6560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:03.405922 6560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:03.407778 6560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "5ddafc84f77c4006afb7cd110c90edfb"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "605a51d2929e72bf8ee66dbf694ea588"
server_key_iv: "b6f895e1f7fd186a1960bb373f37522a"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.408303 6560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "5ddafc84f77c4006afb7cd110c90edfb"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "605a51d2929e72bf8ee66dbf694ea588"
server_key_iv: "b6f895e1f7fd186a1960bb373f37522a"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.411885 6560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 22:43:03.414228 6578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.415355 6560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:03.415481 6560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "5ddafc84f77c4006afb7cd110c90edfb"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "605a51d2929e72bf8ee66dbf694ea588"
server_key_iv: "b6f895e1f7fd186a1960bb373f37522a"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.415578 6560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:03.435583 6560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:03.438582 6560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:03.438766 6560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:03.439355 6560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:03.440207 6560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:03.440275 6560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.440344 6560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:03.440392 6560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.448989 6560 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:45229
I20260810 22:43:03.449028 6711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:45229 every 8 connection(s)
I20260810 22:43:03.449873 6560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:43:03.457996 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 6560
I20260810 22:43:03.458158 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:43:03.458500 20265 external_mini_cluster.cc:1428] Setting key 4a707bf8b8b45895a4cc479543648fa2
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:03.463798 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.451565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:49983 (local address 127.19.202.126:46153)
0810 22:43:03.451689 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.451692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.452430 (+ 738us) server_negotiation.cc:407] Connection header received
0810 22:43:03.453748 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.453751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.453809 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.453902 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:03.454975 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.455503 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.456671 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.456984 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.457097 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.459933 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:03.459939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:03.459943 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:03.459975 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:03.461726 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.462230 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.462237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.462241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.462340 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.462688 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.462692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.462694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.462883 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:03.462993 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.463491 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.463624 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:43:03.464658 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.451859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:46153 (local address 127.19.202.65:49983)
0810 22:43:03.452284 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:03.452324 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:03.453549 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:03.454071 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:03.454078 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:03.454398 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:03.454834 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.454846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.455630 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.455633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.456523 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.456533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.457141 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.457146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.457311 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.457901 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:03.457923 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:03.459751 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:03.461847 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:03.461852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:03.461864 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:03.462133 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:03.462437 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:03.462439 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:03.462440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:03.462584 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:03.462992 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:03.462998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:03.463255 (+ 257us) client_negotiation.cc:769] Sending connection context
0810 22:43:03.463441 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 22:43:03.463785 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260810 22:43:03.465934 6712 heartbeater.cc:347] Connected to a master server at 127.19.202.126:46153
I20260810 22:43:03.466187 6712 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:03.466686 6712 heartbeater.cc:511] Master 127.19.202.126:46153 requested a full tablet report, sending...
I20260810 22:43:03.468138 6489 ts_manager.cc:194] Registered new tserver with Master: 5ddafc84f77c4006afb7cd110c90edfb (127.19.202.65:45229)
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:43:03.469360 6489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:49983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:43:03.487530 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:46153
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:03.598345 6717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:03.598585 6717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:03.598626 6717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:03.602522 6717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:03.602571 6717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:03.602627 6717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:43:03.607767 6717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:46153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:03.608714 6717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:03.609740 6717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:03.619387 6726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:03.619387 6724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:03.619426 6722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:03.619760 6717 server_base.cc:1034] running on GCE node
I20260810 22:43:03.620280 6717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:03.620954 6717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:03.622133 6717 hybrid_clock.cc:648] HybridClock initialized: now 1786401783622113 us; error 30 us; skew 500 ppm
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:03.625068 6717 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:43:03.625952 6717 webserver.cc:460] Webserver started at http://127.19.202.66:33707/ using document root <none> and password file <none>
I20260810 22:43:03.626524 6717 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:03.626616 6717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:03.626848 6717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:03.628976 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "6fab6fbb19884b5db360f6008d271793"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "1a017192dbd683833461fc670647d54b"
server_key_iv: "0c5c7e7d20d3378c93e407a7e6633024"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.629544 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "6fab6fbb19884b5db360f6008d271793"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "1a017192dbd683833461fc670647d54b"
server_key_iv: "0c5c7e7d20d3378c93e407a7e6633024"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.633229 6717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:43:03.635726 6734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.637058 6717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:03.637188 6717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "6fab6fbb19884b5db360f6008d271793"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "1a017192dbd683833461fc670647d54b"
server_key_iv: "0c5c7e7d20d3378c93e407a7e6633024"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.637297 6717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:03.649777 6717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:03.653105 6717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:03.653301 6717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:03.653882 6717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:03.654717 6717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:03.654784 6717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.654857 6717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:03.654897 6717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.663800 6717 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:39385
I20260810 22:43:03.663842 6869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:39385 every 8 connection(s)
I20260810 22:43:03.664734 6717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:43:03.674006 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 6717
I20260810 22:43:03.674153 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:43:03.674521 20265 external_mini_cluster.cc:1428] Setting key 302b5bb8f1fca9a91e4bd64d2c6dff61
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:03.679049 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.666661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:48815 (local address 127.19.202.126:46153)
0810 22:43:03.666794 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.666797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.667635 (+ 838us) server_negotiation.cc:407] Connection header received
0810 22:43:03.669123 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.669126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.669185 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.669287 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:03.670448 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.670976 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.672124 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.672340 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.672447 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.675251 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:03.675255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:03.675257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:03.675282 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:03.676734 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.677333 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.677340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.677344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.677431 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.677761 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.677765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.677767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.678002 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:03.678145 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.678698 (+ 553us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.678838 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:43:03.679798 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.667041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:46153 (local address 127.19.202.66:48815)
0810 22:43:03.667490 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:03.667533 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:03.668920 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:03.669475 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:03.669482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:03.669830 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:03.670302 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.670315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.671103 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.671106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.671981 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.671990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.672496 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.672500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.672667 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.673299 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:03.673317 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:03.675070 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:03.676904 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:03.676910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:03.676924 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:03.677193 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:03.677536 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:03.677539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:03.677540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:03.677666 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:03.678171 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:03.678180 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:03.678457 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 22:43:03.678622 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 22:43:03.678882 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
I20260810 22:43:03.680966 6870 heartbeater.cc:347] Connected to a master server at 127.19.202.126:46153
I20260810 22:43:03.681226 6870 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:03.681739 6870 heartbeater.cc:511] Master 127.19.202.126:46153 requested a full tablet report, sending...
I20260810 22:43:03.682843 6489 ts_manager.cc:194] Registered new tserver with Master: 6fab6fbb19884b5db360f6008d271793 (127.19.202.66:39385)
I20260810 22:43:03.683686 6489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:48815
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:43:03.703207 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:46153
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:03.816883 6876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:03.817121 6876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:03.817188 6876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:03.821326 6876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:03.821376 6876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:03.821467 6876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:43:03.826740 6876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:46153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:03.827795 6876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:03.828878 6876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:03.838738 6883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:03.838738 6886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:03.838778 6884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:03.838801 6876 server_base.cc:1034] running on GCE node
I20260810 22:43:03.839438 6876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:03.840101 6876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:03.841262 6876 hybrid_clock.cc:648] HybridClock initialized: now 1786401783841240 us; error 32 us; skew 500 ppm
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:03.844127 6876 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:43:03.845016 6876 webserver.cc:460] Webserver started at http://127.19.202.67:41613/ using document root <none> and password file <none>
I20260810 22:43:03.845533 6876 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:03.845592 6876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:03.845757 6876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:03.847757 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "dc8f4325060f4007914243103ff7ee06"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "b6086b4329f91f231040606d7a3bf7ed"
server_key_iv: "fc3deb0942971f21af7b09200e0f19b7"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.848271 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "dc8f4325060f4007914243103ff7ee06"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "b6086b4329f91f231040606d7a3bf7ed"
server_key_iv: "fc3deb0942971f21af7b09200e0f19b7"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.852146 6876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 22:43:03.854560 6893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.855764 6876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 22:43:03.855860 6876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "dc8f4325060f4007914243103ff7ee06"
format_stamp: "Formatted at 2026-08-10 22:43:03 on dist-test-slave-1x4g"
server_key: "b6086b4329f91f231040606d7a3bf7ed"
server_key_iv: "fc3deb0942971f21af7b09200e0f19b7"
server_key_version: "encryptionkey@0"
I20260810 22:43:03.855960 6876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:03.879058 6876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:03.882354 6876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:03.882529 6876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:03.883103 6876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:03.883908 6876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:03.883965 6876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.884016 6876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:03.884034 6876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:03.893301 6876 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:45475
I20260810 22:43:03.893326 7019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:45475 every 8 connection(s)
I20260810 22:43:03.894229 6876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:43:03.900462 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 6876
I20260810 22:43:03.900611 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:43:03.900985 20265 external_mini_cluster.cc:1428] Setting key 9c22416903d335093a6a4a475011ddc7
Aug 10 22:43:03 dist-test-slave-1x4g krb5kdc[6438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:03.908485 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.896093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:50713 (local address 127.19.202.126:46153)
0810 22:43:03.896240 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.896244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.897136 (+ 892us) server_negotiation.cc:407] Connection header received
0810 22:43:03.898592 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.898595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.898654 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.898756 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:03.899981 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.900542 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.902001 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.902212 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.902312 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.904927 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:03.904931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:03.904933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:03.904960 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:03.906355 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.906880 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.906884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.906891 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.906967 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.907340 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.907343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.907345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.907538 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:03.907643 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.908171 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.908297 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:43:03.909421 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.896449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:46153 (local address 127.19.202.67:50713)
0810 22:43:03.896989 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:03.897032 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:03.898377 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:03.898938 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:03.898945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:03.899313 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:03.899824 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.899837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.900714 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.900717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.901884 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.901896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.902366 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.902370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.902561 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.903175 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:03.903198 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:03.904735 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:03.906487 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:03.906493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:03.906507 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:03.906770 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:03.907087 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:03.907090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:03.907092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:03.907237 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:03.907658 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:03.907667 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:03.907926 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 22:43:03.908109 (+ 183us) client_negotiation.cc:240] Negotiation successful
0810 22:43:03.908450 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":153,"threads_started":1}
I20260810 22:43:03.910532 7020 heartbeater.cc:347] Connected to a master server at 127.19.202.126:46153
I20260810 22:43:03.910751 7020 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:03.911216 7020 heartbeater.cc:511] Master 127.19.202.126:46153 requested a full tablet report, sending...
I20260810 22:43:03.912130 6489 ts_manager.cc:194] Registered new tserver with Master: dc8f4325060f4007914243103ff7ee06 (127.19.202.67:45475)
I20260810 22:43:03.912680 6489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:50713
I20260810 22:43:03.914769 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:43:03.923736 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.916741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59386 (local address 127.19.202.126:46153)
0810 22:43:03.916862 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.916865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.917064 (+ 199us) server_negotiation.cc:407] Connection header received
0810 22:43:03.917173 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.917175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.917212 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.917271 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:03.917922 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.918413 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.919184 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.919454 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.919534 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.920894 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:03.920897 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:03.920899 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:03.920921 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:03.922381 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.922779 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.922782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.922783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.922856 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:03.923070 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:03.923072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:03.923073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:03.923272 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:03.923379 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.923541 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.923628 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:43:03.926477 6489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:43:03.929534 6489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:43:03.943801 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.938821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.19.202.67:45475)
0810 22:43:03.939282 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.939296 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.939335 (+ 39us) server_negotiation.cc:407] Connection header received
0810 22:43:03.939406 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.939410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.939606 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.939781 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:03.939965 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.941179 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.942601 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.943160 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.943277 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.943426 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.943582 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.943668 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":151,"threads_started":1}
I20260810 22:43:03.943799 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.938014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45475 (local address 127.0.0.1:54642)
0810 22:43:03.938481 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:03.938495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:03.938578 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:03.939756 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:03.939759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:03.939777 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:03.939869 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.939876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.941314 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.941318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.942492 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.942501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.943280 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.943284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.943386 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.943421 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 22:43:03.943541 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 22:43:03.943661 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":59,"spinlock_wait_cycles":4864,"thread_start_us":86,"threads_started":1}
I20260810 22:43:03.944166 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.938314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.19.202.65:45229)
0810 22:43:03.938870 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.938887 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.938930 (+ 43us) server_negotiation.cc:407] Connection header received
0810 22:43:03.939017 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.939022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.939210 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.939383 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:03.939710 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.940675 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.942362 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.943203 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.943303 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.943430 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.943912 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.944014 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":124,"threads_started":1}
I20260810 22:43:03.944226 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.937610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:45229 (local address 127.0.0.1:39740)
0810 22:43:03.937947 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:03.937991 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:03.938102 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:03.939390 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:03.939397 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:03.939422 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:03.939575 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.939585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.940936 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.940939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.942211 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.942221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.943499 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.943503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.943597 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.943624 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 22:43:03.943699 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 22:43:03.943784 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":91,"threads_started":1}
I20260810 22:43:03.944453 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.937707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39385 (local address 127.0.0.1:36676)
0810 22:43:03.938463 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:03.938476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:03.938578 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:03.940084 (+ 1506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:03.940087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:03.940106 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:03.940205 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.940211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.941638 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.941641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.942719 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:03.942730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.943973 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.943977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:03.944066 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.944088 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:43:03.944157 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 22:43:03.944233 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":1,"spinlock_wait_cycles":3712,"thread_start_us":101,"threads_started":1}
I20260810 22:43:03.944559 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:03.938428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36676 (local address 127.19.202.66:39385)
0810 22:43:03.939636 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:03.939670 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:03.939691 (+ 21us) server_negotiation.cc:407] Connection header received
0810 22:43:03.939763 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:03.939767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:03.939947 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:03.940093 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:03.940346 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.941478 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.942861 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:03.943689 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:03.943778 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:03.943913 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:03.944313 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 22:43:03.944416 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108,"thread_start_us":132,"threads_started":1}
I20260810 22:43:03.947091 6790 tablet_service.cc:1467] Processing CreateTablet for tablet a5e3ebc201bb4b1aa09a2e1d96f19887 (DEFAULT_TABLE table=test-table [id=ed22f5568ce74169ae34fe9ab90b0204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:03.947136 6631 tablet_service.cc:1467] Processing CreateTablet for tablet a5e3ebc201bb4b1aa09a2e1d96f19887 (DEFAULT_TABLE table=test-table [id=ed22f5568ce74169ae34fe9ab90b0204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:03.947436 6947 tablet_service.cc:1467] Processing CreateTablet for tablet a5e3ebc201bb4b1aa09a2e1d96f19887 (DEFAULT_TABLE table=test-table [id=ed22f5568ce74169ae34fe9ab90b0204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:03.947973 6790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e3ebc201bb4b1aa09a2e1d96f19887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:03.948125 6631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e3ebc201bb4b1aa09a2e1d96f19887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:03.948256 6947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e3ebc201bb4b1aa09a2e1d96f19887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:03.954135 7037 tablet_bootstrap.cc:492] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Bootstrap starting.
I20260810 22:43:03.955503 7038 tablet_bootstrap.cc:492] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb: Bootstrap starting.
I20260810 22:43:03.956007 7039 tablet_bootstrap.cc:492] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Bootstrap starting.
I20260810 22:43:03.956545 7037 tablet_bootstrap.cc:654] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:03.957775 7038 tablet_bootstrap.cc:654] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:03.957813 7037 log.cc:824] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:03.958124 7039 tablet_bootstrap.cc:654] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:03.959233 7038 log.cc:824] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:03.959447 7039 log.cc:824] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:03.960717 7037 tablet_bootstrap.cc:492] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: No bootstrap required, opened a new log
I20260810 22:43:03.960958 7037 ts_tablet_manager.cc:1397] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 22:43:03.961279 7038 tablet_bootstrap.cc:492] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb: No bootstrap required, opened a new log
I20260810 22:43:03.961422 7039 tablet_bootstrap.cc:492] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: No bootstrap required, opened a new log
I20260810 22:43:03.961483 7038 ts_tablet_manager.cc:1397] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 22:43:03.961583 7039 ts_tablet_manager.cc:1397] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 22:43:03.963903 7037 raft_consensus.cc:348] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:03.964156 7037 raft_consensus.cc:374] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:03.964210 7037 raft_consensus.cc:729] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fab6fbb19884b5db360f6008d271793, State: Initialized, Role: FOLLOWER
I20260810 22:43:03.964658 7037 consensus_queue.cc:260] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:03.965510 6870 heartbeater.cc:503] Master 127.19.202.126:46153 was elected leader, sending a full tablet report...
I20260810 22:43:03.965354 7038 raft_consensus.cc:348] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:03.965603 7038 raft_consensus.cc:374] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:03.965656 7038 raft_consensus.cc:729] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ddafc84f77c4006afb7cd110c90edfb, State: Initialized, Role: FOLLOWER
I20260810 22:43:03.965685 7037 ts_tablet_manager.cc:1428] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:43:03.966073 7038 consensus_queue.cc:260] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:03.965354 7039 raft_consensus.cc:348] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:03.966440 7039 raft_consensus.cc:374] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:03.966497 7039 raft_consensus.cc:729] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc8f4325060f4007914243103ff7ee06, State: Initialized, Role: FOLLOWER
I20260810 22:43:03.966962 7038 ts_tablet_manager.cc:1428] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:43:03.966986 6712 heartbeater.cc:503] Master 127.19.202.126:46153 was elected leader, sending a full tablet report...
I20260810 22:43:03.966929 7039 consensus_queue.cc:260] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:03.967535 7020 heartbeater.cc:503] Master 127.19.202.126:46153 was elected leader, sending a full tablet report...
I20260810 22:43:03.967850 7039 ts_tablet_manager.cc:1428] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:43:04.076280 7043 raft_consensus.cc:482] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:04.076532 7043 raft_consensus.cc:504] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:04.077778 7043 leader_election.cc:288] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ddafc84f77c4006afb7cd110c90edfb (127.19.202.65:45229), dc8f4325060f4007914243103ff7ee06 (127.19.202.67:45475)
I20260810 22:43:04.081792 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.078191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:38283 (local address 127.19.202.67:45475)
0810 22:43:04.078326 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:04.078331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:04.078359 (+ 28us) server_negotiation.cc:407] Connection header received
0810 22:43:04.078460 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:04.078464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:04.078521 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:04.078606 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:04.078965 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.079622 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.080744 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.081205 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.081294 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.081335 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:04.081592 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 22:43:04.081664 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:43:04.081794 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.078054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45475 (local address 127.19.202.66:38283)
0810 22:43:04.078204 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:04.078222 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:04.078326 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:04.078629 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:04.078632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:04.078651 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:04.078822 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.078829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.079733 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.079737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.080599 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.080610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.081323 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.081330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.081457 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.081482 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:43:04.081557 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 22:43:04.081645 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 22:43:04.081820 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.078176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:60107 (local address 127.19.202.65:45229)
0810 22:43:04.078399 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:04.078404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:04.078723 (+ 319us) server_negotiation.cc:407] Connection header received
0810 22:43:04.078858 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:04.078864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:04.078928 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:04.079004 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:04.079291 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.079825 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.080898 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.081298 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.081430 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.081477 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:04.081642 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 22:43:04.081715 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 22:43:04.081809 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.078054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:45229 (local address 127.19.202.66:60107)
0810 22:43:04.078626 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:04.078639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:04.078733 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:04.079030 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:04.079033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:04.079051 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:04.079182 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.079189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.079947 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.079950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.080769 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.080779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.081426 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.081429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.081501 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.081519 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 22:43:04.081611 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 22:43:04.081674 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":133,"thread_start_us":129,"threads_started":1}
I20260810 22:43:04.082441 6655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ebc201bb4b1aa09a2e1d96f19887" candidate_uuid: "6fab6fbb19884b5db360f6008d271793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ddafc84f77c4006afb7cd110c90edfb" is_pre_election: true
I20260810 22:43:04.082533 6968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ebc201bb4b1aa09a2e1d96f19887" candidate_uuid: "6fab6fbb19884b5db360f6008d271793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8f4325060f4007914243103ff7ee06" is_pre_election: true
I20260810 22:43:04.082773 6655 raft_consensus.cc:2454] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fab6fbb19884b5db360f6008d271793 in term 0.
I20260810 22:43:04.082831 6968 raft_consensus.cc:2454] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fab6fbb19884b5db360f6008d271793 in term 0.
I20260810 22:43:04.083295 6737 leader_election.cc:302] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ddafc84f77c4006afb7cd110c90edfb, 6fab6fbb19884b5db360f6008d271793; no voters:
I20260810 22:43:04.083595 7043 raft_consensus.cc:2781] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:43:04.083704 7043 raft_consensus.cc:482] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:43:04.083748 7043 raft_consensus.cc:3037] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:04.084815 7043 raft_consensus.cc:504] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:04.085201 7043 leader_election.cc:288] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [CANDIDATE]: Term 1 election: Requested vote from peers 5ddafc84f77c4006afb7cd110c90edfb (127.19.202.65:45229), dc8f4325060f4007914243103ff7ee06 (127.19.202.67:45475)
I20260810 22:43:04.085525 6655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ebc201bb4b1aa09a2e1d96f19887" candidate_uuid: "6fab6fbb19884b5db360f6008d271793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ddafc84f77c4006afb7cd110c90edfb"
I20260810 22:43:04.085646 6655 raft_consensus.cc:3037] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:04.086786 6655 raft_consensus.cc:2454] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fab6fbb19884b5db360f6008d271793 in term 1.
I20260810 22:43:04.087038 6968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e3ebc201bb4b1aa09a2e1d96f19887" candidate_uuid: "6fab6fbb19884b5db360f6008d271793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8f4325060f4007914243103ff7ee06"
I20260810 22:43:04.087075 6737 leader_election.cc:302] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ddafc84f77c4006afb7cd110c90edfb, 6fab6fbb19884b5db360f6008d271793; no voters:
I20260810 22:43:04.087227 7043 raft_consensus.cc:2781] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:04.087209 6968 raft_consensus.cc:3037] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:04.087482 7043 raft_consensus.cc:686] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 1 LEADER]: Becoming Leader. State: Replica: 6fab6fbb19884b5db360f6008d271793, State: Running, Role: LEADER
I20260810 22:43:04.087821 7043 consensus_queue.cc:237] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } }
I20260810 22:43:04.088363 6968 raft_consensus.cc:2454] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fab6fbb19884b5db360f6008d271793 in term 1.
I20260810 22:43:04.090764 6489 catalog_manager.cc:5495] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fab6fbb19884b5db360f6008d271793 (127.19.202.66). New cstate: current_term: 1 leader_uuid: "6fab6fbb19884b5db360f6008d271793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fab6fbb19884b5db360f6008d271793" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260810 22:43:04.130251 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.127112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.19.202.66:39385)
0810 22:43:04.127247 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:04.127250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:04.127279 (+ 29us) server_negotiation.cc:407] Connection header received
0810 22:43:04.127358 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:04.127360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:04.127426 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:04.127514 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:04.127717 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.128175 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.128989 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.129180 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.129261 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.129501 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:04.129916 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:04.130006 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:04.130069 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 22:43:04.130117 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:43:04.135882 6968 raft_consensus.cc:1261] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 1 FOLLOWER]: Refusing update from remote peer 6fab6fbb19884b5db360f6008d271793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:04.135895 6655 raft_consensus.cc:1261] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 5ddafc84f77c4006afb7cd110c90edfb [term 1 FOLLOWER]: Refusing update from remote peer 6fab6fbb19884b5db360f6008d271793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:04.136605 7043 consensus_queue.cc:1035] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ddafc84f77c4006afb7cd110c90edfb" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 45229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:04.136831 7048 consensus_queue.cc:1035] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc8f4325060f4007914243103ff7ee06" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:04.144308 7053 mvcc.cc:205] Tried to move back new op lower bound from 7317101707813498880 to 7317101707626565632. Current Snapshot: MvccSnapshot[applied={T|T < 7317101707813498880}]
I20260810 22:43:04.145807 7052 mvcc.cc:205] Tried to move back new op lower bound from 7317101707813498880 to 7317101707626565632. Current Snapshot: MvccSnapshot[applied={T|T < 7317101707813498880 or (T in {7317101707813498880})}]
I20260810 22:43:04.151079 6489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:04.151255 6489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:04.153625 6489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48ed5fd8df9342f99127dce1540d4c6c: Sending DeleteTablet for 3 replicas of tablet a5e3ebc201bb4b1aa09a2e1d96f19887
I20260810 22:43:04.154315 6790 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e3ebc201bb4b1aa09a2e1d96f19887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:36676
I20260810 22:43:04.154314 6631 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e3ebc201bb4b1aa09a2e1d96f19887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:39740
I20260810 22:43:04.154486 6947 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e3ebc201bb4b1aa09a2e1d96f19887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:04 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:54642
I20260810 22:43:04.154930 7069 tablet_replica.cc:330] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: stopping tablet replica
I20260810 22:43:04.155189 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 6560
I20260810 22:43:04.155071 7065 tablet_replica.cc:330] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: stopping tablet replica
I20260810 22:43:04.159495 7065 raft_consensus.cc:2229] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:43:04.159843 7065 raft_consensus.cc:2258] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:04.161778 7065 ts_tablet_manager.cc:1905] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:04.164325 7065 ts_tablet_manager.cc:1918] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:04.164441 7065 log.cc:1196] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/a5e3ebc201bb4b1aa09a2e1d96f19887
I20260810 22:43:04.164716 7065 ts_tablet_manager.cc:1939] T a5e3ebc201bb4b1aa09a2e1d96f19887 P 6fab6fbb19884b5db360f6008d271793: Deleting consensus metadata
I20260810 22:43:04.169924 7069 raft_consensus.cc:2229] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:04.170267 7069 raft_consensus.cc:2258] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:04.170521 6474 catalog_manager.cc:4841] TS 6fab6fbb19884b5db360f6008d271793 (127.19.202.66:39385): tablet a5e3ebc201bb4b1aa09a2e1d96f19887 (table test-table [id=ed22f5568ce74169ae34fe9ab90b0204]) successfully deleted
I20260810 22:43:04.171937 7069 ts_tablet_manager.cc:1905] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:04.174544 7069 ts_tablet_manager.cc:1918] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:04.174703 7069 log.cc:1196] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/a5e3ebc201bb4b1aa09a2e1d96f19887
I20260810 22:43:04.175071 7069 ts_tablet_manager.cc:1939] T a5e3ebc201bb4b1aa09a2e1d96f19887 P dc8f4325060f4007914243103ff7ee06: Deleting consensus metadata
I20260810 22:43:04.176285 6475 catalog_manager.cc:4841] TS dc8f4325060f4007914243103ff7ee06 (127.19.202.67:45475): tablet a5e3ebc201bb4b1aa09a2e1d96f19887 (table test-table [id=ed22f5568ce74169ae34fe9ab90b0204]) successfully deleted
W20260810 22:43:04.178930 6473 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:45229 (error 108)
I20260810 22:43:04.179849 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 6717
I20260810 22:43:04.180039 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.179749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:45229 (local address 127.0.0.1:39742)
0810 22:43:04.179897 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:04.179965 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:45229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 22:43:04.183615 6473 catalog_manager.cc:4584] TS 5ddafc84f77c4006afb7cd110c90edfb (127.19.202.65:45229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5e3ebc201bb4b1aa09a2e1d96f19887: Network error: Client connection negotiation failed: client connection to 127.19.202.65:45229: connect: Connection refused (error 111)
I20260810 22:43:04.194440 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 6876
I20260810 22:43:04.203665 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 6452
2026-08-10T22:43:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2452 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T22:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:04Z Disabled control of system clock
I20260810 22:43:04.237337 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:35681
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:35681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 22:43:04.347103 7075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:04.347335 7075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:04.347368 7075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:04.351126 7075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:04.351188 7075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:04.351202 7075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:04.351217 7075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:04.351230 7075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:04.355499 7075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:35681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:35681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:04.356397 7075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:04.357760 7075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:43:04.366146 7075 server_base.cc:1034] running on GCE node
W20260810 22:43:04.366139 7081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:04.366125 7084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:04.366156 7082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:04.366822 7075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:04.367865 7075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:04.369065 7075 hybrid_clock.cc:648] HybridClock initialized: now 1786401784369039 us; error 41 us; skew 500 ppm
I20260810 22:43:04.370796 7075 webserver.cc:460] Webserver started at http://127.19.202.126:42037/ using document root <none> and password file <none>
I20260810 22:43:04.371359 7075 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:04.371443 7075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:04.371685 7075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:04.373797 7075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "6ae1fd6604874a15b3e4e5e926ce1367"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "a2bdbe520e285e65c6458373941994d9"
server_key_iv: "e8d22a1735f2b55d0e21a31213cbe680"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.374379 7075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "6ae1fd6604874a15b3e4e5e926ce1367"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "a2bdbe520e285e65c6458373941994d9"
server_key_iv: "e8d22a1735f2b55d0e21a31213cbe680"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.377986 7075 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:04.380230 7091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.381454 7075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:04.381577 7075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "6ae1fd6604874a15b3e4e5e926ce1367"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "a2bdbe520e285e65c6458373941994d9"
server_key_iv: "e8d22a1735f2b55d0e21a31213cbe680"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.381695 7075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:04.398833 7075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:04.399482 7075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:04.399694 7075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:04.406556 7075 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:35681
I20260810 22:43:04.406564 7152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:35681 every 8 connection(s)
I20260810 22:43:04.407490 7075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:04.410533 7154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:04.413112 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 7075
I20260810 22:43:04.413269 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:04.413573 20265 external_mini_cluster.cc:1428] Setting key 889794782402744fec6fa959be33bef3
I20260810 22:43:04.416880 7154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367: Bootstrap starting.
I20260810 22:43:04.419447 7154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:04.420289 7154 log.cc:824] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:04.422417 7154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367: No bootstrap required, opened a new log
I20260810 22:43:04.422956 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.414990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.19.202.126:35681)
0810 22:43:04.415657 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:04.415672 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:04.415707 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:43:04.416298 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:04.416313 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:04.416737 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:04.417128 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:04.417485 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.418486 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.419688 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.420153 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.420269 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.421241 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:04.421245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:04.421257 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:04.421609 (+ 352us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:04.421661 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:04.421679 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:04.421750 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:04.421902 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 22:43:04.422174 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":165,"threads_started":1}
I20260810 22:43:04.426417 7154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35681 } }
I20260810 22:43:04.426663 7154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:04.426733 7154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae1fd6604874a15b3e4e5e926ce1367, State: Initialized, Role: FOLLOWER
I20260810 22:43:04.427160 7154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35681 } }
I20260810 22:43:04.427294 7154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:04.427378 7154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:04.427521 7154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:04.428611 7154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35681 } }
I20260810 22:43:04.428958 7154 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ae1fd6604874a15b3e4e5e926ce1367; no voters:
I20260810 22:43:04.429206 7154 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:04.429316 7161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:04.429562 7161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae1fd6604874a15b3e4e5e926ce1367, State: Running, Role: LEADER
I20260810 22:43:04.429879 7161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35681 } }
I20260810 22:43:04.430487 7154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:04.431473 7163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae1fd6604874a15b3e4e5e926ce1367. Latest consensus state: current_term: 1 leader_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35681 } } }
I20260810 22:43:04.431612 7163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:04.431965 7170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:04.432489 7162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae1fd6604874a15b3e4e5e926ce1367" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 35681 } } }
I20260810 22:43:04.432598 7162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:04.434577 7170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:04.439531 7170 catalog_manager.cc:1347] Generated new cluster ID: 184d2d96551b4a1396e4cf34af102f97
I20260810 22:43:04.439611 7170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:04.458990 7170 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:04.459944 7170 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:04.465112 7170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367: Generated new TSK 0
I20260810 22:43:04.465812 7170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:04.474677 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:35681
--builtin_ntp_servers=127.19.202.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 22:43:04.586411 7182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:04.586648 7182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:04.586684 7182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:04.590476 7182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:04.590523 7182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:04.590574 7182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:43:04.595219 7182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:35681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:04.596303 7182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:04.597702 7182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:04.607121 7190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:04.607158 7187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:04.607122 7188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:04.607416 7182 server_base.cc:1034] running on GCE node
I20260810 22:43:04.607776 7182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:04.608332 7182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:04.609500 7182 hybrid_clock.cc:648] HybridClock initialized: now 1786401784609477 us; error 44 us; skew 500 ppm
I20260810 22:43:04.611382 7182 webserver.cc:460] Webserver started at http://127.19.202.65:43359/ using document root <none> and password file <none>
I20260810 22:43:04.611937 7182 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:04.612026 7182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:04.612258 7182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:04.614171 7182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "82c0edd1adba4963be7aa1019869af23"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "b7a24c0a3eb621c7bb59f1b70b828e55"
server_key_iv: "fd74fa2c12bcc66b03420a31df9a8856"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.614755 7182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "82c0edd1adba4963be7aa1019869af23"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "b7a24c0a3eb621c7bb59f1b70b828e55"
server_key_iv: "fd74fa2c12bcc66b03420a31df9a8856"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.618340 7182 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:43:04.620627 7197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.621855 7182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:04.622084 7182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "82c0edd1adba4963be7aa1019869af23"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "b7a24c0a3eb621c7bb59f1b70b828e55"
server_key_iv: "fd74fa2c12bcc66b03420a31df9a8856"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.622195 7182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:04.633987 7182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:04.634624 7182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:04.634805 7182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:04.635354 7182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:04.636215 7182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:04.636277 7182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.636335 7182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:04.636382 7182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.645121 7182 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:35769
I20260810 22:43:04.645176 7326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:35769 every 8 connection(s)
I20260810 22:43:04.645990 7182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:43:04.651736 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 7182
I20260810 22:43:04.651862 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:43:04.652132 20265 external_mini_cluster.cc:1428] Setting key 9d886620149c0bed9173db9d21a8a47f
I20260810 22:43:04.654253 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:35681
--builtin_ntp_servers=127.19.202.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 22:43:04.655336 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.647607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:35783 (local address 127.19.202.126:35681)
0810 22:43:04.647741 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:04.647745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:04.648743 (+ 998us) server_negotiation.cc:407] Connection header received
0810 22:43:04.650235 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:04.650239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:04.650298 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:04.650375 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:04.651219 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.651773 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.653108 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.653444 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.653554 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.654882 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:04.654887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:04.654889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:04.654901 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:04.654921 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:04.654929 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:04.654991 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:04.655178 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 22:43:04.655228 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:43:04.656454 7329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.647971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:35681 (local address 127.19.202.65:35783)
0810 22:43:04.648590 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:04.648640 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:04.650021 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:04.650540 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:04.650547 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:04.650625 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:04.651084 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.651096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.651976 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.651979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.653001 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.653011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.653633 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.653637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.653834 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.654574 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:04.654578 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:04.654629 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:04.654726 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:04.655017 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:04.655029 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:04.655039 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 22:43:04.655165 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 22:43:04.655459 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":114,"threads_started":1}
I20260810 22:43:04.657788 7327 heartbeater.cc:347] Connected to a master server at 127.19.202.126:35681
I20260810 22:43:04.658027 7327 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:04.658514 7327 heartbeater.cc:511] Master 127.19.202.126:35681 requested a full tablet report, sending...
I20260810 22:43:04.660246 7110 ts_manager.cc:194] Registered new tserver with Master: 82c0edd1adba4963be7aa1019869af23 (127.19.202.65:35769)
I20260810 22:43:04.662128 7110 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.65:35783
W20260810 22:43:04.769675 7330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:04.769919 7330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:04.769963 7330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:04.773828 7330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:04.773870 7330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:04.773932 7330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:43:04.778482 7330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:35681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:04.779428 7330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:04.780732 7330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:04.790360 7337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:04.790426 7330 server_base.cc:1034] running on GCE node
W20260810 22:43:04.790360 7340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:04.790397 7336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:04.791044 7330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:04.791675 7330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:04.792860 7330 hybrid_clock.cc:648] HybridClock initialized: now 1786401784792839 us; error 28 us; skew 500 ppm
I20260810 22:43:04.794606 7330 webserver.cc:460] Webserver started at http://127.19.202.66:45659/ using document root <none> and password file <none>
I20260810 22:43:04.795178 7330 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:04.795264 7330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:04.795475 7330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:04.797570 7330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "016fcbb87ba8483c9ca235dc565a6f85"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "7f6fbd5681687688c1b0a6695dcc1960"
server_key_iv: "4ed1b042b0739992bcff528912c9db8a"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.798118 7330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "016fcbb87ba8483c9ca235dc565a6f85"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "7f6fbd5681687688c1b0a6695dcc1960"
server_key_iv: "4ed1b042b0739992bcff528912c9db8a"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.801745 7330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:43:04.804144 7346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.805334 7330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:04.805465 7330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "016fcbb87ba8483c9ca235dc565a6f85"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "7f6fbd5681687688c1b0a6695dcc1960"
server_key_iv: "4ed1b042b0739992bcff528912c9db8a"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.805608 7330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:04.827396 7330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:04.828279 7330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:04.828488 7330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:04.829308 7330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:04.830657 7330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:04.830722 7330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.830771 7330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:04.830803 7330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.840142 7330 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:38905
I20260810 22:43:04.840174 7481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:38905 every 8 connection(s)
I20260810 22:43:04.841087 7330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:43:04.842027 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 7330
I20260810 22:43:04.842163 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:43:04.842433 20265 external_mini_cluster.cc:1428] Setting key 5545977cab425ca2eb9a8c4377e6334a
I20260810 22:43:04.844388 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:35681
--builtin_ntp_servers=127.19.202.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 22:43:04.850745 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.842907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:47745 (local address 127.19.202.126:35681)
0810 22:43:04.843072 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:04.843077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:04.844177 (+ 1100us) server_negotiation.cc:407] Connection header received
0810 22:43:04.845714 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:04.845720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:04.845784 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:04.845879 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:04.847086 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.847593 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.848705 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.849049 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.849161 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.850268 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:04.850271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:04.850274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:04.850286 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:04.850306 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:04.850321 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:04.850404 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:04.850546 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 22:43:04.850606 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 22:43:04.851600 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:04.843317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:35681 (local address 127.19.202.66:47745)
0810 22:43:04.844016 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:04.844068 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:04.845513 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:04.846064 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:04.846072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:04.846205 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:04.846928 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.846950 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.847718 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.847722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.848565 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:04.848574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:04.849201 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:04.849205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:04.849375 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:04.850021 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:04.850023 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:04.850061 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:04.850122 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:04.850397 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:04.850406 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:04.850415 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 22:43:04.850544 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 22:43:04.850810 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":143,"threads_started":1}
I20260810 22:43:04.852809 7482 heartbeater.cc:347] Connected to a master server at 127.19.202.126:35681
I20260810 22:43:04.853040 7482 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:04.853526 7482 heartbeater.cc:511] Master 127.19.202.126:35681 requested a full tablet report, sending...
I20260810 22:43:04.854583 7110 ts_manager.cc:194] Registered new tserver with Master: 016fcbb87ba8483c9ca235dc565a6f85 (127.19.202.66:38905)
I20260810 22:43:04.855116 7110 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.66:47745
W20260810 22:43:04.956919 7485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:04.957177 7485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:04.957247 7485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:04.961129 7485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:04.961215 7485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:04.961297 7485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:43:04.966281 7485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:35681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:04.967365 7485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:04.968705 7485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:04.978044 7492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:04.978052 7493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:04.978088 7485 server_base.cc:1034] running on GCE node
W20260810 22:43:04.978053 7495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:04.978664 7485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:04.979267 7485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:04.980427 7485 hybrid_clock.cc:648] HybridClock initialized: now 1786401784980413 us; error 24 us; skew 500 ppm
I20260810 22:43:04.982256 7485 webserver.cc:460] Webserver started at http://127.19.202.67:40149/ using document root <none> and password file <none>
I20260810 22:43:04.982829 7485 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:04.982892 7485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:04.983117 7485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:04.985219 7485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "2f34ca1a8728446a92be34348b6faed2"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "76d6d8b32ba7fce4be3b5c8ad9adb3ee"
server_key_iv: "2ef53169c465b06aa8c859fd03944e23"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.985795 7485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "2f34ca1a8728446a92be34348b6faed2"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "76d6d8b32ba7fce4be3b5c8ad9adb3ee"
server_key_iv: "2ef53169c465b06aa8c859fd03944e23"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.989251 7485 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:43:04.991612 7501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:04.992847 7485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:04.992962 7485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "2f34ca1a8728446a92be34348b6faed2"
format_stamp: "Formatted at 2026-08-10 22:43:04 on dist-test-slave-1x4g"
server_key: "76d6d8b32ba7fce4be3b5c8ad9adb3ee"
server_key_iv: "2ef53169c465b06aa8c859fd03944e23"
server_key_version: "encryptionkey@0"
I20260810 22:43:04.993060 7485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:05.005195 7485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:05.005786 7485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:05.005962 7485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:05.006515 7485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:05.007416 7485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:05.007469 7485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:05.007530 7485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:05.007565 7485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:05.015964 7485 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:39103
I20260810 22:43:05.015992 7640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:39103 every 8 connection(s)
I20260810 22:43:05.016894 7485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:43:05.021450 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 7485
I20260810 22:43:05.021574 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:43:05.021862 20265 external_mini_cluster.cc:1428] Setting key 5cfcf299018dd6ce941176a0f38799c4
I20260810 22:43:05.026175 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.018614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:39647 (local address 127.19.202.126:35681)
0810 22:43:05.018774 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.018779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.019775 (+ 996us) server_negotiation.cc:407] Connection header received
0810 22:43:05.021227 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.021229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.021276 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.021336 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:05.022410 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.023028 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.024416 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.024628 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.024705 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.025789 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:05.025792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:05.025794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:05.025805 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:05.025822 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:05.025829 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:05.025885 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.026024 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.026071 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:43:05.026993 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.019003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:35681 (local address 127.19.202.67:39647)
0810 22:43:05.019639 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:05.019679 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:05.021027 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:05.021518 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:05.021530 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:05.021640 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:05.022282 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.022298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.023148 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.023151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.024311 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.024321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.024751 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.024754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.024962 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.025557 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:05.025559 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:05.025598 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:05.025660 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:05.025896 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:05.025903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:05.025907 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:43:05.026011 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 22:43:05.026246 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":118,"threads_started":1}
I20260810 22:43:05.027945 7641 heartbeater.cc:347] Connected to a master server at 127.19.202.126:35681
I20260810 22:43:05.028170 7641 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:05.028661 7641 heartbeater.cc:511] Master 127.19.202.126:35681 requested a full tablet report, sending...
I20260810 22:43:05.029675 7110 ts_manager.cc:194] Registered new tserver with Master: 2f34ca1a8728446a92be34348b6faed2 (127.19.202.67:39103)
I20260810 22:43:05.030225 7110 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.67:39647
I20260810 22:43:05.035640 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:43:05.041195 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.037661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.19.202.126:35681)
0810 22:43:05.037825 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.037832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.037963 (+ 131us) server_negotiation.cc:407] Connection header received
0810 22:43:05.038065 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.038067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.038105 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.038167 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:05.038427 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.038942 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.039731 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.039954 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.040032 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.040727 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:05.040730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:05.040732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:05.040743 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:05.040761 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:05.040769 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:05.040879 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.040972 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.041018 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 22:43:05.043922 7110 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:43:05.047008 7110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:43:05.061843 7660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.055972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39103 (local address 127.0.0.1:52490)
0810 22:43:05.056749 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:05.056762 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:05.056888 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:05.057400 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:05.057406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:05.057428 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:05.057572 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.057579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.058987 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.058992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.060144 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.060154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.061261 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.061265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.061369 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.061404 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 22:43:05.061515 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 22:43:05.061649 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":141,"spinlock_wait_cycles":12928,"thread_start_us":98,"threads_started":1}
I20260810 22:43:05.062068 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.055470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.19.202.66:38905)
0810 22:43:05.056028 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.056045 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.056740 (+ 695us) server_negotiation.cc:407] Connection header received
0810 22:43:05.057073 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.057079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.057278 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.057428 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:05.057895 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.058971 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.060528 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.061149 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.061268 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.061459 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.061785 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.061899 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":115,"threads_started":1}
I20260810 22:43:05.062189 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.056139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52490 (local address 127.19.202.67:39103)
0810 22:43:05.056513 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.056524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.056862 (+ 338us) server_negotiation.cc:407] Connection header received
0810 22:43:05.057099 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.057103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.057254 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.057411 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:05.057691 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.058871 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.060277 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.061127 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.061234 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.061408 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.061930 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.062039 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":103,"threads_started":1}
I20260810 22:43:05.062245 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.055229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:38905 (local address 127.0.0.1:39874)
0810 22:43:05.056609 (+ 1380us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:05.056650 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:05.056888 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:05.057578 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:05.057581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:05.057607 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:05.057766 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.057785 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.059064 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.059068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.060409 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.060426 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.061483 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.061486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.061577 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.061615 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:43:05.061724 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 22:43:05.061798 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1199,"mutex_wait_us":68,"spinlock_wait_cycles":2048,"thread_start_us":101,"threads_started":1}
I20260810 22:43:05.063900 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.055160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35769 (local address 127.0.0.1:40542)
0810 22:43:05.056608 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:05.056647 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:05.056893 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:05.057578 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:05.057581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:05.057609 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:05.057761 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.057768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.060403 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.060406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.061518 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.061526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.063486 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.063493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.063590 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.063618 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 22:43:05.063705 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 22:43:05.063772 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":31,"spinlock_wait_cycles":19456,"thread_start_us":99,"threads_started":1}
I20260810 22:43:05.064113 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.055295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.19.202.65:35769)
0810 22:43:05.055944 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.055959 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.056749 (+ 790us) server_negotiation.cc:407] Connection header received
0810 22:43:05.057102 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.057106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.057277 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.057428 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:05.059105 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.060285 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.061694 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.063341 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.063458 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.063623 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.063730 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.063816 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":169,"threads_started":1}
I20260810 22:43:05.065308 7558 tablet_service.cc:1467] Processing CreateTablet for tablet 8cfd5bbd532f472aad84d63ab047929b (DEFAULT_TABLE table=test-table [id=4a8196a9e824476ebbc0deab14ca115b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:05.065421 7400 tablet_service.cc:1467] Processing CreateTablet for tablet 8cfd5bbd532f472aad84d63ab047929b (DEFAULT_TABLE table=test-table [id=4a8196a9e824476ebbc0deab14ca115b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:05.066500 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cfd5bbd532f472aad84d63ab047929b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:05.066504 7558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cfd5bbd532f472aad84d63ab047929b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:05.066591 7250 tablet_service.cc:1467] Processing CreateTablet for tablet 8cfd5bbd532f472aad84d63ab047929b (DEFAULT_TABLE table=test-table [id=4a8196a9e824476ebbc0deab14ca115b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:05.067493 7250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cfd5bbd532f472aad84d63ab047929b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:05.072321 7662 tablet_bootstrap.cc:492] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Bootstrap starting.
I20260810 22:43:05.074525 7664 tablet_bootstrap.cc:492] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23: Bootstrap starting.
I20260810 22:43:05.074944 7662 tablet_bootstrap.cc:654] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:05.075661 7662 log.cc:824] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:05.076290 7664 tablet_bootstrap.cc:654] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:05.077052 7664 log.cc:824] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:05.077497 7662 tablet_bootstrap.cc:492] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: No bootstrap required, opened a new log
I20260810 22:43:05.077683 7662 ts_tablet_manager.cc:1397] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:43:05.079320 7664 tablet_bootstrap.cc:492] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23: No bootstrap required, opened a new log
I20260810 22:43:05.079479 7664 ts_tablet_manager.cc:1397] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 22:43:05.076329 7663 tablet_bootstrap.cc:492] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Bootstrap starting.
I20260810 22:43:05.081302 7662 raft_consensus.cc:348] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.081583 7662 raft_consensus.cc:374] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:05.081640 7662 raft_consensus.cc:729] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f34ca1a8728446a92be34348b6faed2, State: Initialized, Role: FOLLOWER
I20260810 22:43:05.082113 7662 consensus_queue.cc:260] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.082509 7664 raft_consensus.cc:348] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.082762 7664 raft_consensus.cc:374] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:05.082800 7641 heartbeater.cc:503] Master 127.19.202.126:35681 was elected leader, sending a full tablet report...
I20260810 22:43:05.082827 7664 raft_consensus.cc:729] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c0edd1adba4963be7aa1019869af23, State: Initialized, Role: FOLLOWER
I20260810 22:43:05.083109 7663 tablet_bootstrap.cc:654] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:05.083264 7664 consensus_queue.cc:260] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.084017 7327 heartbeater.cc:503] Master 127.19.202.126:35681 was elected leader, sending a full tablet report...
I20260810 22:43:05.084221 7662 ts_tablet_manager.cc:1428] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:43:05.084285 7664 ts_tablet_manager.cc:1428] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260810 22:43:05.084841 7663 log.cc:824] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:05.086907 7663 tablet_bootstrap.cc:492] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: No bootstrap required, opened a new log
I20260810 22:43:05.087106 7663 ts_tablet_manager.cc:1397] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260810 22:43:05.090405 7663 raft_consensus.cc:348] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.090618 7663 raft_consensus.cc:374] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:05.090658 7663 raft_consensus.cc:729] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016fcbb87ba8483c9ca235dc565a6f85, State: Initialized, Role: FOLLOWER
I20260810 22:43:05.091032 7663 consensus_queue.cc:260] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.091621 7482 heartbeater.cc:503] Master 127.19.202.126:35681 was elected leader, sending a full tablet report...
I20260810 22:43:05.091876 7663 ts_tablet_manager.cc:1428] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:43:05.144687 7669 raft_consensus.cc:482] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:05.144976 7669 raft_consensus.cc:504] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.146186 7669 leader_election.cc:288] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016fcbb87ba8483c9ca235dc565a6f85 (127.19.202.66:38905), 2f34ca1a8728446a92be34348b6faed2 (127.19.202.67:39103)
W20260810 22:43:05.147111 7328 tablet.cc:2369] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:43:05.150139 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.146542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:44581 (local address 127.19.202.66:38905)
0810 22:43:05.146669 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.146674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.146691 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:43:05.146839 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.146841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.146914 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.147009 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:05.147315 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.147874 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.149127 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.149550 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.149669 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.149727 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.149919 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.150001 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:43:05.150146 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.146632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:52219 (local address 127.19.202.67:39103)
0810 22:43:05.146782 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.146787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.146923 (+ 136us) server_negotiation.cc:407] Connection header received
0810 22:43:05.147020 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.147022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.147067 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.147131 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:05.147468 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.148079 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.149190 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.149580 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.149709 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.149762 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.149964 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.150043 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 22:43:05.150144 7329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.146420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:38905 (local address 127.19.202.65:44581)
0810 22:43:05.146601 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:05.146616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:05.146711 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:05.147049 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:05.147052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:05.147068 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:05.147196 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.147203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.148006 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.148009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.148994 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.149005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.149679 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.149683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.149787 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.149813 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 22:43:05.149876 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 22:43:05.149979 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":55}
I20260810 22:43:05.150153 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.146440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:39103 (local address 127.19.202.65:52219)
0810 22:43:05.146827 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:05.146844 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:05.146911 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:05.147148 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:05.147151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:05.147169 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:05.147342 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.147349 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.148198 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.148201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.149038 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:05.149048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.149724 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.149727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:05.149818 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.149835 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 22:43:05.149936 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 22:43:05.150001 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":21,"thread_start_us":90,"threads_started":1}
I20260810 22:43:05.151072 7583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfd5bbd532f472aad84d63ab047929b" candidate_uuid: "82c0edd1adba4963be7aa1019869af23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f34ca1a8728446a92be34348b6faed2" is_pre_election: true
I20260810 22:43:05.151074 7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfd5bbd532f472aad84d63ab047929b" candidate_uuid: "82c0edd1adba4963be7aa1019869af23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016fcbb87ba8483c9ca235dc565a6f85" is_pre_election: true
I20260810 22:43:05.151343 7424 raft_consensus.cc:2454] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82c0edd1adba4963be7aa1019869af23 in term 0.
I20260810 22:43:05.151355 7583 raft_consensus.cc:2454] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82c0edd1adba4963be7aa1019869af23 in term 0.
I20260810 22:43:05.151819 7202 leader_election.cc:302] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f34ca1a8728446a92be34348b6faed2, 82c0edd1adba4963be7aa1019869af23; no voters:
I20260810 22:43:05.152078 7669 raft_consensus.cc:2781] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:43:05.152161 7669 raft_consensus.cc:482] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:43:05.152220 7669 raft_consensus.cc:3037] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:05.153353 7669 raft_consensus.cc:504] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.153676 7669 leader_election.cc:288] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [CANDIDATE]: Term 1 election: Requested vote from peers 016fcbb87ba8483c9ca235dc565a6f85 (127.19.202.66:38905), 2f34ca1a8728446a92be34348b6faed2 (127.19.202.67:39103)
I20260810 22:43:05.153980 7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfd5bbd532f472aad84d63ab047929b" candidate_uuid: "82c0edd1adba4963be7aa1019869af23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016fcbb87ba8483c9ca235dc565a6f85"
I20260810 22:43:05.154031 7583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfd5bbd532f472aad84d63ab047929b" candidate_uuid: "82c0edd1adba4963be7aa1019869af23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f34ca1a8728446a92be34348b6faed2"
I20260810 22:43:05.154109 7424 raft_consensus.cc:3037] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:05.154145 7583 raft_consensus.cc:3037] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:05.155159 7424 raft_consensus.cc:2454] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82c0edd1adba4963be7aa1019869af23 in term 1.
I20260810 22:43:05.155249 7583 raft_consensus.cc:2454] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82c0edd1adba4963be7aa1019869af23 in term 1.
I20260810 22:43:05.155485 7200 leader_election.cc:302] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016fcbb87ba8483c9ca235dc565a6f85, 82c0edd1adba4963be7aa1019869af23; no voters:
I20260810 22:43:05.155655 7669 raft_consensus.cc:2781] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:05.155894 7669 raft_consensus.cc:686] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [term 1 LEADER]: Becoming Leader. State: Replica: 82c0edd1adba4963be7aa1019869af23, State: Running, Role: LEADER
I20260810 22:43:05.156213 7669 consensus_queue.cc:237] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } }
I20260810 22:43:05.159621 7110 catalog_manager.cc:5495] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82c0edd1adba4963be7aa1019869af23 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "82c0edd1adba4963be7aa1019869af23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c0edd1adba4963be7aa1019869af23" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 35769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 } health_report { overall_health: UNKNOWN } } }
I20260810 22:43:05.190992 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.187338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.19.202.65:35769)
0810 22:43:05.187480 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:05.187485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:05.187501 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:43:05.187702 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:05.187704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:05.187763 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:05.187874 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:05.188112 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.188606 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.189497 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:05.189734 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:05.189841 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:05.190147 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:05.190562 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:05.190694 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:05.190758 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 22:43:05.190812 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:43:05.196939 7583 raft_consensus.cc:1261] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 1 FOLLOWER]: Refusing update from remote peer 82c0edd1adba4963be7aa1019869af23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:05.196962 7424 raft_consensus.cc:1261] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 1 FOLLOWER]: Refusing update from remote peer 82c0edd1adba4963be7aa1019869af23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:05.197654 7669 consensus_queue.cc:1035] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016fcbb87ba8483c9ca235dc565a6f85" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 38905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:05.197882 7674 consensus_queue.cc:1035] T 8cfd5bbd532f472aad84d63ab047929b P 82c0edd1adba4963be7aa1019869af23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f34ca1a8728446a92be34348b6faed2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 39103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:05.206991 7679 mvcc.cc:205] Tried to move back new op lower bound from 7317101712158949376 to 7317101712002834432. Current Snapshot: MvccSnapshot[applied={T|T < 7317101712158949376}]
I20260810 22:43:05.208686 7678 mvcc.cc:205] Tried to move back new op lower bound from 7317101712158949376 to 7317101712002834432. Current Snapshot: MvccSnapshot[applied={T|T < 7317101712158949376}]
I20260810 22:43:05.213075 7109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:05.213250 7109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:05.215652 7109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ae1fd6604874a15b3e4e5e926ce1367: Sending DeleteTablet for 3 replicas of tablet 8cfd5bbd532f472aad84d63ab047929b
I20260810 22:43:05.216449 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cfd5bbd532f472aad84d63ab047929b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:05 UTC) from {username='slave'} at 127.0.0.1:39874
I20260810 22:43:05.216511 7558 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cfd5bbd532f472aad84d63ab047929b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:05 UTC) from {username='slave'} at 127.0.0.1:52490
I20260810 22:43:05.217025 7687 tablet_replica.cc:330] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: stopping tablet replica
I20260810 22:43:05.217163 7250 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cfd5bbd532f472aad84d63ab047929b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:05 UTC) from {username='slave'} at 127.0.0.1:40542
I20260810 22:43:05.217235 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 7182
I20260810 22:43:05.218767 7686 tablet_replica.cc:330] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: stopping tablet replica
I20260810 22:43:05.218998 7686 raft_consensus.cc:2229] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:05.219261 7686 raft_consensus.cc:2258] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:05.219345 7687 raft_consensus.cc:2229] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:05.219661 7687 raft_consensus.cc:2258] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:05.221647 7687 ts_tablet_manager.cc:1905] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:05.221686 7686 ts_tablet_manager.cc:1905] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:05.224445 7686 ts_tablet_manager.cc:1918] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:05.224436 7687 ts_tablet_manager.cc:1918] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:05.224547 7687 log.cc:1196] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/8cfd5bbd532f472aad84d63ab047929b
I20260810 22:43:05.224547 7686 log.cc:1196] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/8cfd5bbd532f472aad84d63ab047929b
I20260810 22:43:05.224916 7686 ts_tablet_manager.cc:1939] T 8cfd5bbd532f472aad84d63ab047929b P 016fcbb87ba8483c9ca235dc565a6f85: Deleting consensus metadata
I20260810 22:43:05.224898 7687 ts_tablet_manager.cc:1939] T 8cfd5bbd532f472aad84d63ab047929b P 2f34ca1a8728446a92be34348b6faed2: Deleting consensus metadata
I20260810 22:43:05.226186 7093 catalog_manager.cc:4841] TS 016fcbb87ba8483c9ca235dc565a6f85 (127.19.202.66:38905): tablet 8cfd5bbd532f472aad84d63ab047929b (table test-table [id=4a8196a9e824476ebbc0deab14ca115b]) successfully deleted
I20260810 22:43:05.226230 7095 catalog_manager.cc:4841] TS 2f34ca1a8728446a92be34348b6faed2 (127.19.202.67:39103): tablet 8cfd5bbd532f472aad84d63ab047929b (table test-table [id=4a8196a9e824476ebbc0deab14ca115b]) successfully deleted
W20260810 22:43:05.227415 7092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:35769 (error 108)
I20260810 22:43:05.227484 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 7330
I20260810 22:43:05.231377 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:05.231118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:35769 (local address 127.0.0.1:40562)
0810 22:43:05.231239 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:05.231304 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:43:05.231557 7092 catalog_manager.cc:4584] TS 82c0edd1adba4963be7aa1019869af23 (127.19.202.65:35769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8cfd5bbd532f472aad84d63ab047929b: Network error: Client connection negotiation failed: client connection to 127.19.202.65:35769: connect: Connection refused (error 111)
I20260810 22:43:05.235831 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 7485
I20260810 22:43:05.242791 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 7075
2026-08-10T22:43:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1037 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:05 dist-test-slave-1x4g krb5kdc[7692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:05 dist-test-slave-1x4g krb5kdc[7692](info): set up 2 sockets
Aug 10 22:43:05 dist-test-slave-1x4g krb5kdc[7692](info): commencing operation
krb5kdc: starting...
W20260810 22:43:06.312458 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:06 dist-test-slave-1x4g krb5kdc[7692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:06.437572 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34451
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:34451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:06.546891 7706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:06.547147 7706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:06.547214 7706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:06.551518 7706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:06.551596 7706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:06.551633 7706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:06.551669 7706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:06.551702 7706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:06.556843 7706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:34451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:34451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:06.557862 7706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:06.559159 7706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:06.567766 7712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:06.567777 7715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:06.567785 7713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:06.567966 7706 server_base.cc:1034] running on GCE node
I20260810 22:43:06.568595 7706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:06.569672 7706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:06.570851 7706 hybrid_clock.cc:648] HybridClock initialized: now 1786401786570821 us; error 47 us; skew 500 ppm
Aug 10 22:43:06 dist-test-slave-1x4g krb5kdc[7692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401786, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:06.573729 7706 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:06.574558 7706 webserver.cc:460] Webserver started at http://127.19.202.126:44377/ using document root <none> and password file <none>
I20260810 22:43:06.575125 7706 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:06.575183 7706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:06.575349 7706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:06.577428 7706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "f52c7408f62e4ce592ac0025c005b65d"
format_stamp: "Formatted at 2026-08-10 22:43:06 on dist-test-slave-1x4g"
server_key: "ca31ef908438a1a020d6952ecc0a1982"
server_key_iv: "22efcdf850d0e8b886896a3aca4edc92"
server_key_version: "encryptionkey@0"
I20260810 22:43:06.577937 7706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "f52c7408f62e4ce592ac0025c005b65d"
format_stamp: "Formatted at 2026-08-10 22:43:06 on dist-test-slave-1x4g"
server_key: "ca31ef908438a1a020d6952ecc0a1982"
server_key_iv: "22efcdf850d0e8b886896a3aca4edc92"
server_key_version: "encryptionkey@0"
I20260810 22:43:06.581820 7706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 22:43:06.584229 7724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:06.585493 7706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:06.585623 7706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "f52c7408f62e4ce592ac0025c005b65d"
format_stamp: "Formatted at 2026-08-10 22:43:06 on dist-test-slave-1x4g"
server_key: "ca31ef908438a1a020d6952ecc0a1982"
server_key_iv: "22efcdf850d0e8b886896a3aca4edc92"
server_key_version: "encryptionkey@0"
I20260810 22:43:06.585728 7706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:06.595728 7706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:06.598767 7706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:06.598985 7706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:06.606755 7706 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:34451
I20260810 22:43:06.606858 7787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:34451 every 8 connection(s)
I20260810 22:43:06.607781 7706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:06.610859 7788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:06.614230 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 7706
I20260810 22:43:06.614388 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:06.614702 20265 external_mini_cluster.cc:1428] Setting key e01bc5baae128b8a0afcbf04e62033a8
I20260810 22:43:06.617190 7788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d: Bootstrap starting.
I20260810 22:43:06.619563 7788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:06.620314 7788 log.cc:824] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:06.622254 7788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d: No bootstrap required, opened a new log
Aug 10 22:43:06 dist-test-slave-1x4g krb5kdc[7692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:06.626562 7788 raft_consensus.cc:348] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52c7408f62e4ce592ac0025c005b65d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34451 } }
I20260810 22:43:06.626852 7788 raft_consensus.cc:374] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:06.626942 7788 raft_consensus.cc:729] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52c7408f62e4ce592ac0025c005b65d, State: Initialized, Role: FOLLOWER
I20260810 22:43:06.627411 7788 consensus_queue.cc:260] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52c7408f62e4ce592ac0025c005b65d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34451 } }
I20260810 22:43:06.627604 7788 raft_consensus.cc:388] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:06.627676 7788 raft_consensus.cc:482] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:06.627774 7788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:06.628945 7788 raft_consensus.cc:504] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52c7408f62e4ce592ac0025c005b65d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34451 } }
I20260810 22:43:06.629294 7788 leader_election.cc:302] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f52c7408f62e4ce592ac0025c005b65d; no voters:
I20260810 22:43:06.629612 7788 leader_election.cc:288] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:06.629756 7795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:06.629827 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:06.616186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.19.202.126:34451)
0810 22:43:06.616731 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:06.616750 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:06.616859 (+ 109us) server_negotiation.cc:407] Connection header received
0810 22:43:06.617557 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:06.617576 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:06.617908 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:06.618248 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:06.619111 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:06.620198 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:06.621178 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:06.621646 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:06.621819 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:06.624606 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:06.624612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:06.624625 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:06.624651 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:06.626468 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:06.627008 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:06.627015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:06.627022 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:06.627162 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:06.627483 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:06.627486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:06.627488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:06.628013 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:06.628214 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:06.628483 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 22:43:06.628818 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":214,"threads_started":1}
I20260810 22:43:06.630018 7795 raft_consensus.cc:686] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [term 1 LEADER]: Becoming Leader. State: Replica: f52c7408f62e4ce592ac0025c005b65d, State: Running, Role: LEADER
I20260810 22:43:06.630300 7795 consensus_queue.cc:237] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52c7408f62e4ce592ac0025c005b65d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34451 } }
I20260810 22:43:06.630975 7788 sys_catalog.cc:564] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:06.631549 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f52c7408f62e4ce592ac0025c005b65d. Latest consensus state: current_term: 1 leader_uuid: "f52c7408f62e4ce592ac0025c005b65d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52c7408f62e4ce592ac0025c005b65d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34451 } } }
I20260810 22:43:06.631913 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:06.631773 7797 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f52c7408f62e4ce592ac0025c005b65d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52c7408f62e4ce592ac0025c005b65d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 34451 } } }
I20260810 22:43:06.632174 7797 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:06.632517 7804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:06.635157 7804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:06.640710 7804 catalog_manager.cc:1347] Generated new cluster ID: b3b40cc49902421a9e8a4a3530f3020b
I20260810 22:43:06.640813 7804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:06.648669 7804 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:06.649523 7804 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:06.655550 7804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d: Generated new TSK 0
I20260810 22:43:06.656173 7804 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:43:06.692142 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34451
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:06.809896 7820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:06.810140 7820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:06.810182 7820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:06.814143 7820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:06.814193 7820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:06.814293 7820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:43:06.819548 7820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:06.820573 7820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:06.821620 7820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:06.831626 7825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:06.831732 7826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:06.831750 7828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:06.832602 7820 server_base.cc:1034] running on GCE node
I20260810 22:43:06.833318 7820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:06.834123 7820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:06.835335 7820 hybrid_clock.cc:648] HybridClock initialized: now 1786401786835308 us; error 42 us; skew 500 ppm
Aug 10 22:43:06 dist-test-slave-1x4g krb5kdc[7692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401786, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:06.842815 7820 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:43:06.843755 7820 webserver.cc:460] Webserver started at http://127.19.202.65:45997/ using document root <none> and password file <none>
I20260810 22:43:06.844362 7820 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:06.844455 7820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:06.844695 7820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:06.846828 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "b42d7ca8ee3243f085327dbd7a5a733b"
format_stamp: "Formatted at 2026-08-10 22:43:06 on dist-test-slave-1x4g"
server_key: "95d3b68e27ee790afc58761c85a56f73"
server_key_iv: "fcd37371296cf8044a6455065b5620f5"
server_key_version: "encryptionkey@0"
I20260810 22:43:06.847405 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "b42d7ca8ee3243f085327dbd7a5a733b"
format_stamp: "Formatted at 2026-08-10 22:43:06 on dist-test-slave-1x4g"
server_key: "95d3b68e27ee790afc58761c85a56f73"
server_key_iv: "fcd37371296cf8044a6455065b5620f5"
server_key_version: "encryptionkey@0"
I20260810 22:43:06.851431 7820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 22:43:06.853899 7836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:06.855049 7820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:06.855194 7820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "b42d7ca8ee3243f085327dbd7a5a733b"
format_stamp: "Formatted at 2026-08-10 22:43:06 on dist-test-slave-1x4g"
server_key: "95d3b68e27ee790afc58761c85a56f73"
server_key_iv: "fcd37371296cf8044a6455065b5620f5"
server_key_version: "encryptionkey@0"
I20260810 22:43:06.855301 7820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:06.869240 7820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:06.871945 7820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:06.872152 7820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:06.872716 7820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:06.873557 7820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:06.873628 7820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:06.873698 7820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:06.873740 7820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:06.883000 7820 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:46457
I20260810 22:43:06.883041 7962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:46457 every 8 connection(s)
I20260810 22:43:06.883894 7820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:43:06.889396 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 7820
I20260810 22:43:06.889546 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:43:06.889936 20265 external_mini_cluster.cc:1428] Setting key bff99ca40dc45320d6725c36af8f4559
Aug 10 22:43:06 dist-test-slave-1x4g krb5kdc[7692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401786, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:06.899431 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:06.885643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:57275 (local address 127.19.202.126:34451)
0810 22:43:06.885813 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:06.885818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:06.886636 (+ 818us) server_negotiation.cc:407] Connection header received
0810 22:43:06.887898 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:06.887900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:06.887955 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:06.888028 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:06.889373 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:06.890142 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:06.891700 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:06.891971 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:06.892070 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:06.895391 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:06.895397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:06.895403 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:06.895435 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:06.897178 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:06.897681 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:06.897687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:06.897691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:06.897792 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:06.898144 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:06.898150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:06.898152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:06.898414 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:06.898525 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:06.899049 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 22:43:06.899203 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 22:43:06.900182 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:06.886030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34451 (local address 127.19.202.65:57275)
0810 22:43:06.886473 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:06.886517 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:06.887707 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:06.888215 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:06.888222 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:06.888566 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:06.889208 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:06.889232 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:06.890303 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:06.890307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:06.891574 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:06.891586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:06.892148 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:06.892154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:06.892389 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:06.893086 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:06.893111 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:06.895191 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:06.897299 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:06.897304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:06.897318 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:06.897593 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:06.897914 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:06.897917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:06.897920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:06.898048 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:06.898531 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:06.898536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:06.898791 (+ 255us) client_negotiation.cc:769] Sending connection context
0810 22:43:06.898962 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 22:43:06.899216 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":108,"threads_started":1}
I20260810 22:43:06.901638 7963 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34451
I20260810 22:43:06.901904 7963 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:43:06.902544 7963 heartbeater.cc:511] Master 127.19.202.126:34451 requested a full tablet report, sending...
I20260810 22:43:06.904052 7746 ts_manager.cc:194] Registered new tserver with Master: b42d7ca8ee3243f085327dbd7a5a733b (127.19.202.65:46457)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:43:06.905386 7746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:57275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:43:06.921535 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34451
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:07.034466 7968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:07.034708 7968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:07.034751 7968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:07.038743 7968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:07.038793 7968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:07.038865 7968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:43:07.043883 7968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:07.044914 7968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:07.045883 7968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:07.055335 7976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:07.055336 7977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:07.055435 7979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:07.055666 7968 server_base.cc:1034] running on GCE node
I20260810 22:43:07.056084 7968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:07.056725 7968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:07.057936 7968 hybrid_clock.cc:648] HybridClock initialized: now 1786401787057895 us; error 43 us; skew 500 ppm
Aug 10 22:43:07 dist-test-slave-1x4g krb5kdc[7692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401787, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:07.060817 7968 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:43:07.061630 7968 webserver.cc:460] Webserver started at http://127.19.202.66:44479/ using document root <none> and password file <none>
I20260810 22:43:07.062155 7968 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:07.062225 7968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:07.062381 7968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:07.064375 7968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "5c66361a52d04fe29519c8746f92fe2f"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "c54d44e94b8cf5eedcde6efc09621418"
server_key_iv: "57c2155b6ef6c425ca51a2db9bbae24d"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.065021 7968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "5c66361a52d04fe29519c8746f92fe2f"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "c54d44e94b8cf5eedcde6efc09621418"
server_key_iv: "57c2155b6ef6c425ca51a2db9bbae24d"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.068859 7968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 22:43:07.071208 7986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:07.072408 7968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 22:43:07.072525 7968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "5c66361a52d04fe29519c8746f92fe2f"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "c54d44e94b8cf5eedcde6efc09621418"
server_key_iv: "57c2155b6ef6c425ca51a2db9bbae24d"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.072607 7968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:07.087469 7968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:07.090246 7968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:07.090402 7968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:07.090972 7968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:07.091768 7968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:07.091813 7968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:07.091851 7968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:07.091866 7968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:07.100876 7968 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:44057
I20260810 22:43:07.100888 8121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:44057 every 8 connection(s)
I20260810 22:43:07.101848 7968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:43:07.108336 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 7968
I20260810 22:43:07.108470 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:43:07.108808 20265 external_mini_cluster.cc:1428] Setting key ef676ec361a6dfc4f6f444d623483e32
Aug 10 22:43:07 dist-test-slave-1x4g krb5kdc[7692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401787, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:07.116230 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.103560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:57215 (local address 127.19.202.126:34451)
0810 22:43:07.103702 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.103706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.104525 (+ 819us) server_negotiation.cc:407] Connection header received
0810 22:43:07.105916 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.105920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.105979 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.106096 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:07.107260 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.107795 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.109219 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.109513 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.109642 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.112409 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:07.112412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:07.112414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:07.112445 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:07.114012 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:07.114529 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:07.114536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:07.114540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:07.114637 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:07.114966 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:07.114972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:07.114976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:07.115247 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:07.115353 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.115847 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.116000 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:43:07.117007 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.103934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34451 (local address 127.19.202.66:57215)
0810 22:43:07.104377 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.104419 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:07.105698 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:07.106275 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:07.106282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:07.106625 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:07.107109 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.107122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.107943 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.107948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.109095 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.109106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.109678 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.109683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.109855 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.110499 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:07.110523 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:07.112213 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:07.114120 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:07.114124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:07.114135 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:07.114415 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:07.114735 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:07.114737 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:07.114738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:07.114865 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:07.115350 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:07.115355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:07.115580 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 22:43:07.115748 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 22:43:07.116071 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260810 22:43:07.118183 8122 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34451
I20260810 22:43:07.118422 8122 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:07.118891 8122 heartbeater.cc:511] Master 127.19.202.126:34451 requested a full tablet report, sending...
I20260810 22:43:07.119930 7746 ts_manager.cc:194] Registered new tserver with Master: 5c66361a52d04fe29519c8746f92fe2f (127.19.202.66:44057)
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:43:07.120601 7746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:57215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:43:07.140100 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34451
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:07.253788 8128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:07.254040 8128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:07.254129 8128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:07.258186 8128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:07.258239 8128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:07.258323 8128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:43:07.263751 8128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:34451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:07.264844 8128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:07.265856 8128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:07.275674 8133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:07.275724 8134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:07.275712 8136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:07.275957 8128 server_base.cc:1034] running on GCE node
I20260810 22:43:07.276410 8128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:07.277125 8128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:07.278311 8128 hybrid_clock.cc:648] HybridClock initialized: now 1786401787278299 us; error 34 us; skew 500 ppm
Aug 10 22:43:07 dist-test-slave-1x4g krb5kdc[7692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401787, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:07.281057 8128 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:43:07.281885 8128 webserver.cc:460] Webserver started at http://127.19.202.67:38585/ using document root <none> and password file <none>
I20260810 22:43:07.282459 8128 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:07.282557 8128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:07.282789 8128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:07.285359 8128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "6845f6265a0d4614bfaae75c902a35dc"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "0d090e73dd7a6f3aadd7835c0d1cf438"
server_key_iv: "e259ea40af9c2f80fe5273a68fbd2339"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.286130 8128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "6845f6265a0d4614bfaae75c902a35dc"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "0d090e73dd7a6f3aadd7835c0d1cf438"
server_key_iv: "e259ea40af9c2f80fe5273a68fbd2339"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.290172 8128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 22:43:07.292536 8143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:07.293812 8128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:07.293939 8128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "6845f6265a0d4614bfaae75c902a35dc"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "0d090e73dd7a6f3aadd7835c0d1cf438"
server_key_iv: "e259ea40af9c2f80fe5273a68fbd2339"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.294067 8128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:07.322120 8128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:07.325153 8128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:07.325382 8128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:07.326028 8128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:07.326859 8128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:07.326928 8128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:07.326993 8128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:07.327047 8128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:07.336663 8128 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:36785
I20260810 22:43:07.336736 8268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:36785 every 8 connection(s)
I20260810 22:43:07.337639 8128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:43:07.347090 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 8128
I20260810 22:43:07.347259 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
Aug 10 22:43:07 dist-test-slave-1x4g krb5kdc[7692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401787, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:07.347604 20265 external_mini_cluster.cc:1428] Setting key 27232459f750451087fda9762736de12
I20260810 22:43:07.351724 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.339464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:35811 (local address 127.19.202.126:34451)
0810 22:43:07.339616 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.339619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.340448 (+ 829us) server_negotiation.cc:407] Connection header received
0810 22:43:07.341915 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.341918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.341978 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.342090 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:07.343271 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.343815 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.345157 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.345378 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.345482 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.348160 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:07.348164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:07.348167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:07.348194 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:07.349857 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:07.350325 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:07.350328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:07.350329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:07.350400 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:07.350690 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:07.350692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:07.350693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:07.350861 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:07.350937 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.351445 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.351563 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 22:43:07.352425 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.339819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:34451 (local address 127.19.202.67:35811)
0810 22:43:07.340294 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.340337 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:07.341672 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:07.342284 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:07.342291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:07.342647 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:07.343131 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.343143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.343964 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.343967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.345025 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.345034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.345535 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.345539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.345702 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.346300 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:07.346316 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:07.347994 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:07.349976 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:07.349980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:07.349991 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:07.350241 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:07.350481 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:07.350484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:07.350485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:07.350613 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:07.350963 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:07.350967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:07.351186 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 22:43:07.351336 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 22:43:07.351576 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
I20260810 22:43:07.353608 8269 heartbeater.cc:347] Connected to a master server at 127.19.202.126:34451
I20260810 22:43:07.353847 8269 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:07.354341 8269 heartbeater.cc:511] Master 127.19.202.126:34451 requested a full tablet report, sending...
I20260810 22:43:07.355340 7746 ts_manager.cc:194] Registered new tserver with Master: 6845f6265a0d4614bfaae75c902a35dc (127.19.202.67:36785)
I20260810 22:43:07.355906 7746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:35811
I20260810 22:43:07.361361 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:43:07.370579 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.363331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.19.202.126:34451)
0810 22:43:07.363445 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.363448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.363662 (+ 214us) server_negotiation.cc:407] Connection header received
0810 22:43:07.363751 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.363753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.363787 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.363850 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:07.364489 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.365093 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.365883 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.366091 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.366163 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.367800 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:07.367803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:07.367805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:07.367826 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:07.369069 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:07.369547 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:07.369550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:07.369551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:07.369619 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:07.369882 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:07.369884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:07.369885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:07.370069 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:07.370183 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.370392 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.370466 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:43:07.373329 7746 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:43:07.376433 7746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:43:07.391330 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.385764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:36785 (local address 127.0.0.1:56706)
0810 22:43:07.386265 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.386278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:07.386360 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:07.387047 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:07.387050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:07.387068 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:07.387157 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.387165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.388177 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.388184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.389626 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.389637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.390792 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.390796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.390900 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.390937 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 22:43:07.391057 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 22:43:07.391175 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":40,"thread_start_us":86,"threads_started":1}
I20260810 22:43:07.391336 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.386219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.19.202.67:36785)
0810 22:43:07.386606 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.386618 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.386640 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:43:07.386716 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.386723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.386908 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.387070 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:07.387277 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.388038 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.389776 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.390650 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.390829 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.390981 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.391087 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.391176 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":147,"threads_started":1}
I20260810 22:43:07.391542 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.385347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:46457 (local address 127.0.0.1:49934)
0810 22:43:07.385843 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.385896 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:07.386039 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:07.387187 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:07.387200 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:07.387220 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:07.387295 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.387300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.388822 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.388825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.390047 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.390058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.391125 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.391129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.391212 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.391234 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:43:07.391318 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:43:07.391391 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":2,"spinlock_wait_cycles":8704,"thread_start_us":123,"threads_started":1}
I20260810 22:43:07.391556 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.385544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.19.202.65:46457)
0810 22:43:07.386738 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.386753 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.386775 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:43:07.386851 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.386856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.387044 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.387439 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:07.387572 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.388661 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.390183 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.390985 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.391125 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.391255 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.391344 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.391430 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1091,"thread_start_us":91,"threads_started":1}
I20260810 22:43:07.392074 8281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.385054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:44057 (local address 127.0.0.1:50134)
0810 22:43:07.385844 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.385896 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:07.386039 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:07.386876 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:07.386882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:07.386904 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:07.387030 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.387037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.388550 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.388553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.389785 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.389793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.391704 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.391711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.391805 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.391826 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 22:43:07.391889 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 22:43:07.391949 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":36,"spinlock_wait_cycles":3072,"thread_start_us":101,"threads_started":1}
I20260810 22:43:07.392561 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.385184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50134 (local address 127.19.202.66:44057)
0810 22:43:07.385971 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.385983 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.386009 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:43:07.386251 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.386258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.386495 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.386685 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:07.387173 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.388399 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.389904 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.390680 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.390789 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.390942 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.392293 (+ 1351us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.392408 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":205,"threads_started":1}
I20260810 22:43:07.394825 8198 tablet_service.cc:1467] Processing CreateTablet for tablet 2c23e25eea04464084ca018cee1812c1 (DEFAULT_TABLE table=test-table [id=455156d1fc0f4044b44960a001d5cf0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:07.395031 8041 tablet_service.cc:1467] Processing CreateTablet for tablet 2c23e25eea04464084ca018cee1812c1 (DEFAULT_TABLE table=test-table [id=455156d1fc0f4044b44960a001d5cf0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:07.395123 7889 tablet_service.cc:1467] Processing CreateTablet for tablet 2c23e25eea04464084ca018cee1812c1 (DEFAULT_TABLE table=test-table [id=455156d1fc0f4044b44960a001d5cf0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:07.395870 8041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c23e25eea04464084ca018cee1812c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:07.395931 7889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c23e25eea04464084ca018cee1812c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:07.395931 8198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c23e25eea04464084ca018cee1812c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:07.402577 8287 tablet_bootstrap.cc:492] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Bootstrap starting.
I20260810 22:43:07.403836 8288 tablet_bootstrap.cc:492] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b: Bootstrap starting.
I20260810 22:43:07.404912 8287 tablet_bootstrap.cc:654] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:07.405128 8289 tablet_bootstrap.cc:492] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Bootstrap starting.
I20260810 22:43:07.405817 8288 tablet_bootstrap.cc:654] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:07.406178 8287 log.cc:824] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:07.406654 8288 log.cc:824] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:07.407289 8289 tablet_bootstrap.cc:654] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:07.408128 8287 tablet_bootstrap.cc:492] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: No bootstrap required, opened a new log
I20260810 22:43:07.408342 8287 ts_tablet_manager.cc:1397] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 22:43:07.408541 8289 log.cc:824] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:07.408604 8288 tablet_bootstrap.cc:492] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b: No bootstrap required, opened a new log
I20260810 22:43:07.408790 8288 ts_tablet_manager.cc:1397] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 22:43:07.410482 8289 tablet_bootstrap.cc:492] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: No bootstrap required, opened a new log
I20260810 22:43:07.410665 8289 ts_tablet_manager.cc:1397] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:43:07.412254 8288 raft_consensus.cc:348] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.412302 8287 raft_consensus.cc:348] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.412505 8288 raft_consensus.cc:374] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:07.412556 8288 raft_consensus.cc:729] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b42d7ca8ee3243f085327dbd7a5a733b, State: Initialized, Role: FOLLOWER
I20260810 22:43:07.412570 8287 raft_consensus.cc:374] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:07.412628 8287 raft_consensus.cc:729] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c66361a52d04fe29519c8746f92fe2f, State: Initialized, Role: FOLLOWER
I20260810 22:43:07.413048 8288 consensus_queue.cc:260] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.413060 8287 consensus_queue.cc:260] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.413713 7963 heartbeater.cc:503] Master 127.19.202.126:34451 was elected leader, sending a full tablet report...
I20260810 22:43:07.414043 8288 ts_tablet_manager.cc:1428] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:43:07.414053 8287 ts_tablet_manager.cc:1428] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:43:07.414475 8122 heartbeater.cc:503] Master 127.19.202.126:34451 was elected leader, sending a full tablet report...
I20260810 22:43:07.415328 8289 raft_consensus.cc:348] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.415599 8289 raft_consensus.cc:374] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:07.415655 8289 raft_consensus.cc:729] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6845f6265a0d4614bfaae75c902a35dc, State: Initialized, Role: FOLLOWER
I20260810 22:43:07.416441 8289 consensus_queue.cc:260] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.417435 8289 ts_tablet_manager.cc:1428] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 22:43:07.417683 8269 heartbeater.cc:503] Master 127.19.202.126:34451 was elected leader, sending a full tablet report...
W20260810 22:43:07.588697 8270 tablet.cc:2369] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:43:07.603075 8123 tablet.cc:2369] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:43:07.635231 7964 tablet.cc:2369] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:43:07.651681 8294 raft_consensus.cc:482] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:07.651913 8294 raft_consensus.cc:504] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.653126 8294 leader_election.cc:288] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c66361a52d04fe29519c8746f92fe2f (127.19.202.66:44057), 6845f6265a0d4614bfaae75c902a35dc (127.19.202.67:36785)
I20260810 22:43:07.657472 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.653510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:52265 (local address 127.19.202.66:44057)
0810 22:43:07.653653 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.653656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.653823 (+ 167us) server_negotiation.cc:407] Connection header received
0810 22:43:07.653995 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.654000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.654062 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.654159 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:07.654558 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.655234 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.656358 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.656844 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.656945 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.656986 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.657254 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.657350 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:43:07.657497 8297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.653424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:44057 (local address 127.19.202.65:52265)
0810 22:43:07.653726 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.653742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:07.653858 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:07.654188 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:07.654191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:07.654216 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:07.654419 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.654426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.655344 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.655347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.656228 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.656238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.656954 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.656958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.657079 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.657146 (+ 67us) client_negotiation.cc:769] Sending connection context
0810 22:43:07.657218 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 22:43:07.657285 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":22,"spinlock_wait_cycles":29440,"thread_start_us":108,"threads_started":1}
I20260810 22:43:07.657560 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.653845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:36785 (local address 127.19.202.65:33883)
0810 22:43:07.654172 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.654185 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:07.654312 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:07.654594 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:07.654597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:07.654614 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:07.654725 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.654731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.655530 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.655532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.656266 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:07.656275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.657093 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.657096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:07.657237 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.657262 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:43:07.657339 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 22:43:07.657404 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":17,"spinlock_wait_cycles":768,"thread_start_us":100,"threads_started":1}
I20260810 22:43:07.658000 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.653972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:33883 (local address 127.19.202.67:36785)
0810 22:43:07.654150 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.654155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.654262 (+ 107us) server_negotiation.cc:407] Connection header received
0810 22:43:07.654414 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.654417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.654477 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.654571 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:07.654858 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.655419 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.656409 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.656974 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.657092 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.657148 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.657370 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.657443 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 22:43:07.658279 8065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e25eea04464084ca018cee1812c1" candidate_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c66361a52d04fe29519c8746f92fe2f" is_pre_election: true
I20260810 22:43:07.658634 8065 raft_consensus.cc:2454] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b42d7ca8ee3243f085327dbd7a5a733b in term 0.
I20260810 22:43:07.658602 8220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e25eea04464084ca018cee1812c1" candidate_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6845f6265a0d4614bfaae75c902a35dc" is_pre_election: true
I20260810 22:43:07.658867 8220 raft_consensus.cc:2454] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b42d7ca8ee3243f085327dbd7a5a733b in term 0.
I20260810 22:43:07.659161 7837 leader_election.cc:302] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c66361a52d04fe29519c8746f92fe2f, b42d7ca8ee3243f085327dbd7a5a733b; no voters:
I20260810 22:43:07.659425 8294 raft_consensus.cc:2781] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:43:07.659513 8294 raft_consensus.cc:482] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:43:07.659559 8294 raft_consensus.cc:3037] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:07.660663 8294 raft_consensus.cc:504] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.661048 8294 leader_election.cc:288] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [CANDIDATE]: Term 1 election: Requested vote from peers 5c66361a52d04fe29519c8746f92fe2f (127.19.202.66:44057), 6845f6265a0d4614bfaae75c902a35dc (127.19.202.67:36785)
I20260810 22:43:07.661442 8065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e25eea04464084ca018cee1812c1" candidate_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c66361a52d04fe29519c8746f92fe2f"
I20260810 22:43:07.661434 8220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e25eea04464084ca018cee1812c1" candidate_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6845f6265a0d4614bfaae75c902a35dc"
I20260810 22:43:07.661561 8220 raft_consensus.cc:3037] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:07.661561 8065 raft_consensus.cc:3037] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:07.662842 8220 raft_consensus.cc:2454] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b42d7ca8ee3243f085327dbd7a5a733b in term 1.
I20260810 22:43:07.662842 8065 raft_consensus.cc:2454] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b42d7ca8ee3243f085327dbd7a5a733b in term 1.
I20260810 22:43:07.663154 7837 leader_election.cc:302] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c66361a52d04fe29519c8746f92fe2f, b42d7ca8ee3243f085327dbd7a5a733b; no voters:
I20260810 22:43:07.663307 8294 raft_consensus.cc:2781] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:07.663522 8294 raft_consensus.cc:686] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [term 1 LEADER]: Becoming Leader. State: Replica: b42d7ca8ee3243f085327dbd7a5a733b, State: Running, Role: LEADER
I20260810 22:43:07.663864 8294 consensus_queue.cc:237] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } }
I20260810 22:43:07.667378 7745 catalog_manager.cc:5495] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b reported cstate change: term changed from 0 to 1, leader changed from <none> to b42d7ca8ee3243f085327dbd7a5a733b (127.19.202.65). New cstate: current_term: 1 leader_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b42d7ca8ee3243f085327dbd7a5a733b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 46457 } health_report { overall_health: HEALTHY } } }
I20260810 22:43:07.681141 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.677918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.19.202.65:46457)
0810 22:43:07.678034 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.678037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.678049 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:43:07.678182 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.678184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.678233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.678315 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:07.678530 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.679056 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.679827 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.680035 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.680107 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.680369 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:07.680746 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:07.680928 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.681008 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.681042 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:43:07.686812 8220 raft_consensus.cc:1261] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 1 FOLLOWER]: Refusing update from remote peer b42d7ca8ee3243f085327dbd7a5a733b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:07.686841 8065 raft_consensus.cc:1261] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 1 FOLLOWER]: Refusing update from remote peer b42d7ca8ee3243f085327dbd7a5a733b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:07.687511 8294 consensus_queue.cc:1035] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c66361a52d04fe29519c8746f92fe2f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:07.687690 8299 consensus_queue.cc:1035] T 2c23e25eea04464084ca018cee1812c1 P b42d7ca8ee3243f085327dbd7a5a733b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6845f6265a0d4614bfaae75c902a35dc" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 36785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:07.705458 7746 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:07.705651 7746 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:07.708132 7746 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f52c7408f62e4ce592ac0025c005b65d: Sending DeleteTablet for 3 replicas of tablet 2c23e25eea04464084ca018cee1812c1
I20260810 22:43:07.709069 7889 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c23e25eea04464084ca018cee1812c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:07 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:49934
I20260810 22:43:07.709098 8041 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c23e25eea04464084ca018cee1812c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:07 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:50134
I20260810 22:43:07.709249 8198 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c23e25eea04464084ca018cee1812c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:07 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:56706
I20260810 22:43:07.709602 8314 tablet_replica.cc:330] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: stopping tablet replica
I20260810 22:43:07.709650 8315 tablet_replica.cc:330] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: stopping tablet replica
I20260810 22:43:07.709934 8314 raft_consensus.cc:2229] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:07.709949 8315 raft_consensus.cc:2229] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:07.710259 8314 raft_consensus.cc:2258] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:07.710261 8315 raft_consensus.cc:2258] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:07.710734 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 7820
I20260810 22:43:07.712126 8314 ts_tablet_manager.cc:1905] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:07.712245 8315 ts_tablet_manager.cc:1905] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:07.714458 8314 ts_tablet_manager.cc:1918] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:07.714573 8314 log.cc:1196] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/2c23e25eea04464084ca018cee1812c1
I20260810 22:43:07.714869 8314 ts_tablet_manager.cc:1939] T 2c23e25eea04464084ca018cee1812c1 P 5c66361a52d04fe29519c8746f92fe2f: Deleting consensus metadata
I20260810 22:43:07.715656 8315 ts_tablet_manager.cc:1918] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:07.715749 8315 log.cc:1196] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/2c23e25eea04464084ca018cee1812c1
I20260810 22:43:07.716100 8315 ts_tablet_manager.cc:1939] T 2c23e25eea04464084ca018cee1812c1 P 6845f6265a0d4614bfaae75c902a35dc: Deleting consensus metadata
I20260810 22:43:07.716215 7725 catalog_manager.cc:4841] TS 5c66361a52d04fe29519c8746f92fe2f (127.19.202.66:44057): tablet 2c23e25eea04464084ca018cee1812c1 (table test-table [id=455156d1fc0f4044b44960a001d5cf0a]) successfully deleted
I20260810 22:43:07.717217 7725 catalog_manager.cc:4841] TS 6845f6265a0d4614bfaae75c902a35dc (127.19.202.67:36785): tablet 2c23e25eea04464084ca018cee1812c1 (table test-table [id=455156d1fc0f4044b44960a001d5cf0a]) successfully deleted
W20260810 22:43:07.722839 7728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:46457 (error 108)
I20260810 22:43:07.723090 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 7968
I20260810 22:43:07.725533 8281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.724472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:46457 (local address 127.0.0.1:49956)
0810 22:43:07.725346 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:07.725436 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":759}
W20260810 22:43:07.725718 7728 catalog_manager.cc:4584] TS b42d7ca8ee3243f085327dbd7a5a733b (127.19.202.65:46457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c23e25eea04464084ca018cee1812c1: Network error: Client connection negotiation failed: client connection to 127.19.202.65:46457: connect: Connection refused (error 111)
I20260810 22:43:07.730974 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 8128
I20260810 22:43:07.738947 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 7706
2026-08-10T22:43:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2496 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T22:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:07Z Disabled control of system clock
I20260810 22:43:07.771811 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38209
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 22:43:07.884537 8322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:07.884809 8322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:07.884891 8322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:07.888844 8322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:07.888896 8322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:07.888911 8322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:07.888954 8322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:07.888983 8322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:07.893652 8322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:38209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:07.894608 8322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:07.895853 8322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:07.904349 8332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:07.904385 8328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:07.904618 8322 server_base.cc:1034] running on GCE node
W20260810 22:43:07.904349 8329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:07.905247 8322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:07.906304 8322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:07.907493 8322 hybrid_clock.cc:648] HybridClock initialized: now 1786401787907483 us; error 28 us; skew 500 ppm
I20260810 22:43:07.909394 8322 webserver.cc:460] Webserver started at http://127.19.202.126:45495/ using document root <none> and password file <none>
I20260810 22:43:07.909968 8322 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:07.910058 8322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:07.910283 8322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:07.912300 8322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "22188e37472044a7b765982c959cacf4"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "9fe703655e3f5e0e7bbebcd7a53fd1ef"
server_key_iv: "333280e7bd79fef8352e6beffafe2582"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.912942 8322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "22188e37472044a7b765982c959cacf4"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "9fe703655e3f5e0e7bbebcd7a53fd1ef"
server_key_iv: "333280e7bd79fef8352e6beffafe2582"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.916656 8322 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:43:07.918941 8339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:07.920126 8322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:07.920265 8322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "22188e37472044a7b765982c959cacf4"
format_stamp: "Formatted at 2026-08-10 22:43:07 on dist-test-slave-1x4g"
server_key: "9fe703655e3f5e0e7bbebcd7a53fd1ef"
server_key_iv: "333280e7bd79fef8352e6beffafe2582"
server_key_version: "encryptionkey@0"
I20260810 22:43:07.920377 8322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:07.931371 8322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:07.931995 8322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:07.932181 8322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:07.939137 8322 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:38209
I20260810 22:43:07.939141 8399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:38209 every 8 connection(s)
I20260810 22:43:07.940040 8322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:07.943154 8400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:07.947547 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 8322
I20260810 22:43:07.947680 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:07.947990 20265 external_mini_cluster.cc:1428] Setting key b5cd294f74157424519496fd8f15fbc5
I20260810 22:43:07.948985 8400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4: Bootstrap starting.
I20260810 22:43:07.951179 8400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:07.952033 8400 log.cc:824] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:07.954205 8400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4: No bootstrap required, opened a new log
I20260810 22:43:07.957573 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:07.949402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.19.202.126:38209)
0810 22:43:07.950022 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:07.950038 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:07.950072 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:43:07.950662 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:07.950677 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:07.951068 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:07.951390 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:07.951722 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.952977 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.954123 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:07.954616 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:07.954732 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:07.955558 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:07.955563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:07.955576 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:07.955947 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:07.955997 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:07.956017 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:07.956090 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:07.956248 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 22:43:07.956442 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":124,"threads_started":1}
I20260810 22:43:07.957633 8400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22188e37472044a7b765982c959cacf4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38209 } }
I20260810 22:43:07.957930 8400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:07.957973 8400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22188e37472044a7b765982c959cacf4, State: Initialized, Role: FOLLOWER
I20260810 22:43:07.958391 8400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22188e37472044a7b765982c959cacf4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38209 } }
I20260810 22:43:07.958527 8400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:07.958598 8400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:07.958690 8400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:07.959754 8400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22188e37472044a7b765982c959cacf4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38209 } }
I20260810 22:43:07.960078 8400 leader_election.cc:302] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22188e37472044a7b765982c959cacf4; no voters:
I20260810 22:43:07.960310 8400 leader_election.cc:288] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:07.960431 8406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:07.960691 8406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [term 1 LEADER]: Becoming Leader. State: Replica: 22188e37472044a7b765982c959cacf4, State: Running, Role: LEADER
I20260810 22:43:07.961024 8406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22188e37472044a7b765982c959cacf4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38209 } }
I20260810 22:43:07.961478 8400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:07.962716 8408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22188e37472044a7b765982c959cacf4. Latest consensus state: current_term: 1 leader_uuid: "22188e37472044a7b765982c959cacf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22188e37472044a7b765982c959cacf4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38209 } } }
I20260810 22:43:07.962746 8407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22188e37472044a7b765982c959cacf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22188e37472044a7b765982c959cacf4" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38209 } } }
I20260810 22:43:07.962846 8408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:07.962849 8407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:07.963207 8417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:07.966224 8417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:07.971585 8417 catalog_manager.cc:1347] Generated new cluster ID: 162c7649bf034dcb8521b5dff5a4576f
I20260810 22:43:07.971654 8417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:07.986138 8417 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:07.987197 8417 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:07.996970 8417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4: Generated new TSK 0
I20260810 22:43:07.997591 8417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:08.004907 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38209
--builtin_ntp_servers=127.19.202.84:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 22:43:08.132254 8430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:08.132503 8430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:08.132539 8430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:08.136351 8430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:08.136399 8430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:08.136451 8430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:43:08.141028 8430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:08.142046 8430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:08.143342 8430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:08.152861 8439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:08.152899 8441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:08.152844 8438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:08.153100 8430 server_base.cc:1034] running on GCE node
I20260810 22:43:08.153545 8430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:08.154161 8430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:08.155368 8430 hybrid_clock.cc:648] HybridClock initialized: now 1786401788155358 us; error 38 us; skew 500 ppm
I20260810 22:43:08.157163 8430 webserver.cc:460] Webserver started at http://127.19.202.65:38819/ using document root <none> and password file <none>
I20260810 22:43:08.157725 8430 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:08.157831 8430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:08.158061 8430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:08.160043 8430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "d279a43b11974ab6a24525ca714ec479"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "dbc55b686a9b67f6362ded4438393785"
server_key_iv: "3bfffd57678f08b7394664a39a37cf69"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.160602 8430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "d279a43b11974ab6a24525ca714ec479"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "dbc55b686a9b67f6362ded4438393785"
server_key_iv: "3bfffd57678f08b7394664a39a37cf69"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.164477 8430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 22:43:08.166865 8448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.167982 8430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:08.168124 8430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "d279a43b11974ab6a24525ca714ec479"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "dbc55b686a9b67f6362ded4438393785"
server_key_iv: "3bfffd57678f08b7394664a39a37cf69"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.168231 8430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:08.181634 8430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:08.182221 8430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:08.182403 8430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:08.182977 8430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:08.183856 8430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:08.183903 8430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.183987 8430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:08.184015 8430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.192610 8430 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:41935
I20260810 22:43:08.192651 8588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:41935 every 8 connection(s)
I20260810 22:43:08.193524 8430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:43:08.202780 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.195295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:52087 (local address 127.19.202.126:38209)
0810 22:43:08.195436 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.195445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.196419 (+ 974us) server_negotiation.cc:407] Connection header received
0810 22:43:08.197982 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.197986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.198046 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.198149 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:08.199048 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.199608 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.200765 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.201050 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.201159 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.202273 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:08.202277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:08.202279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:08.202291 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:08.202317 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:08.202325 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:08.202399 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.202550 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.202612 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:43:08.203024 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 8430
I20260810 22:43:08.203176 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:43:08.203480 20265 external_mini_cluster.cc:1428] Setting key f1ef714240b14ddc1c07c76e12131daf
I20260810 22:43:08.203864 8591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.195657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38209 (local address 127.19.202.65:52087)
0810 22:43:08.196268 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.196307 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.197748 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.198341 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.198348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.198433 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:08.198899 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.198911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.199747 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.199755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.200633 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.200643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.201213 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.201217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.201405 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.202028 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:08.202030 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:08.202065 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:08.202124 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:08.202407 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:08.202417 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:08.202424 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.202548 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.202882 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260810 22:43:08.205117 8589 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38209
I20260810 22:43:08.205350 8589 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:08.205581 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38209
--builtin_ntp_servers=127.19.202.84:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 22:43:08.205901 8589 heartbeater.cc:511] Master 127.19.202.126:38209 requested a full tablet report, sending...
I20260810 22:43:08.207404 8358 ts_manager.cc:194] Registered new tserver with Master: d279a43b11974ab6a24525ca714ec479 (127.19.202.65:41935)
I20260810 22:43:08.208539 8358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.65:52087
W20260810 22:43:08.322494 8592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:08.322717 8592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:08.322777 8592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:08.326910 8592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:08.326963 8592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:08.327016 8592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:43:08.331732 8592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:08.332768 8592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:08.334097 8592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:08.343683 8602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:08.343704 8599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:08.343747 8600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:08.343727 8592 server_base.cc:1034] running on GCE node
I20260810 22:43:08.344352 8592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:08.345043 8592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:08.346199 8592 hybrid_clock.cc:648] HybridClock initialized: now 1786401788346166 us; error 52 us; skew 500 ppm
I20260810 22:43:08.348021 8592 webserver.cc:460] Webserver started at http://127.19.202.66:38341/ using document root <none> and password file <none>
I20260810 22:43:08.348542 8592 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:08.348603 8592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:08.348766 8592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:08.350848 8592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "3c9ee8463de74c49adc87ab85f09ce38"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "57114c51f02be941217de38cbe3fceed"
server_key_iv: "cc99e259db7ec3ad544d84d2ce7e172f"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.351447 8592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "3c9ee8463de74c49adc87ab85f09ce38"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "57114c51f02be941217de38cbe3fceed"
server_key_iv: "cc99e259db7ec3ad544d84d2ce7e172f"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.355402 8592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260810 22:43:08.357797 8610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.358932 8592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:08.359069 8592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "3c9ee8463de74c49adc87ab85f09ce38"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "57114c51f02be941217de38cbe3fceed"
server_key_iv: "cc99e259db7ec3ad544d84d2ce7e172f"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.359180 8592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:08.379752 8592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:08.380429 8592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:08.380630 8592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:08.381290 8592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:08.382233 8592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:08.382308 8592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.382376 8592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:08.382426 8592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.391149 8592 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:39265
I20260810 22:43:08.391239 8741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:39265 every 8 connection(s)
I20260810 22:43:08.392071 8592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:43:08.392855 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 8592
I20260810 22:43:08.393028 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:43:08.393327 20265 external_mini_cluster.cc:1428] Setting key 7d3b667bda01c36b0b57c9a69415e4c7
I20260810 22:43:08.395572 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38209
--builtin_ntp_servers=127.19.202.84:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 22:43:08.402961 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.395104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:60331 (local address 127.19.202.126:38209)
0810 22:43:08.395222 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.395228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.395474 (+ 246us) server_negotiation.cc:407] Connection header received
0810 22:43:08.397037 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.397043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.397108 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.397204 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:08.398459 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.398992 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.400853 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.401070 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.401149 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.402494 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:08.402497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:08.402499 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:08.402510 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:08.402528 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:08.402535 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:08.402591 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.402801 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.402849 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:43:08.404269 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.394564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38209 (local address 127.19.202.66:60331)
0810 22:43:08.395312 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.395364 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.396765 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.397436 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.397449 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.397564 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:08.398298 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.398319 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.399129 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.399136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.400691 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.400707 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.401237 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.401244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.401487 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.402178 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:08.402182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:08.402242 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:08.402334 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:08.402626 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:08.402638 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:08.402648 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.402790 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.403133 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":136,"threads_started":1}
I20260810 22:43:08.405471 8742 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38209
I20260810 22:43:08.405754 8742 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:08.406430 8742 heartbeater.cc:511] Master 127.19.202.126:38209 requested a full tablet report, sending...
I20260810 22:43:08.407618 8358 ts_manager.cc:194] Registered new tserver with Master: 3c9ee8463de74c49adc87ab85f09ce38 (127.19.202.66:39265)
I20260810 22:43:08.408186 8358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.66:60331
W20260810 22:43:08.511250 8745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:08.511504 8745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:08.511579 8745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:08.515466 8745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:08.515547 8745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:08.515628 8745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:43:08.520187 8745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:08.521234 8745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:08.522593 8745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:08.531917 8754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:08.531899 8751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:08.531862 8752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:08.531982 8745 server_base.cc:1034] running on GCE node
I20260810 22:43:08.532526 8745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:08.533195 8745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:08.534381 8745 hybrid_clock.cc:648] HybridClock initialized: now 1786401788534370 us; error 25 us; skew 500 ppm
I20260810 22:43:08.536191 8745 webserver.cc:460] Webserver started at http://127.19.202.67:41459/ using document root <none> and password file <none>
I20260810 22:43:08.536831 8745 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:08.536904 8745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:08.537154 8745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:08.539260 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "c4029ed684a346d9881a71a2fb9bd935"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "edc64086ca0c9cfbc0357e4ac74daa25"
server_key_iv: "dfe04a7f90d20e62aed8b37a7b08562f"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.539845 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "c4029ed684a346d9881a71a2fb9bd935"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "edc64086ca0c9cfbc0357e4ac74daa25"
server_key_iv: "dfe04a7f90d20e62aed8b37a7b08562f"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.543515 8745 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 22:43:08.545939 8761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.547186 8745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:08.547349 8745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "c4029ed684a346d9881a71a2fb9bd935"
format_stamp: "Formatted at 2026-08-10 22:43:08 on dist-test-slave-1x4g"
server_key: "edc64086ca0c9cfbc0357e4ac74daa25"
server_key_iv: "dfe04a7f90d20e62aed8b37a7b08562f"
server_key_version: "encryptionkey@0"
I20260810 22:43:08.547464 8745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:08.558732 8745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:08.559397 8745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:08.559587 8745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:08.560182 8745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:08.561195 8745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:08.561266 8745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.561337 8745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:08.561398 8745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:08.570930 8745 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:45307
I20260810 22:43:08.570964 8890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:45307 every 8 connection(s)
I20260810 22:43:08.571888 8745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:43:08.572984 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 8745
I20260810 22:43:08.573110 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:43:08.573402 20265 external_mini_cluster.cc:1428] Setting key c7ec6aace026b6d1ea1f5460ed67800f
I20260810 22:43:08.581310 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.575128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:40599 (local address 127.19.202.126:38209)
0810 22:43:08.575279 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.575284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.575301 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:43:08.576601 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.576626 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.576680 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.576759 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:08.577632 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.578173 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.579340 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.579575 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.579685 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.580798 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:08.580803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:08.580806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:08.580818 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:08.580839 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:08.580854 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:08.580934 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.581102 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.581159 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 22:43:08.582084 8894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.574359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:38209 (local address 127.19.202.67:40599)
0810 22:43:08.574995 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.575034 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.576393 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.576954 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.576961 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.577040 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:08.577500 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.577511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.578309 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.578312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.579204 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.579213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.579727 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.579731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.579893 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.580528 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:08.580530 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:08.580569 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:08.580628 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:08.580940 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:08.580951 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:08.580960 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.581085 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.581358 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":115,"threads_started":1}
I20260810 22:43:08.583094 8891 heartbeater.cc:347] Connected to a master server at 127.19.202.126:38209
I20260810 22:43:08.583325 8891 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:08.583812 8891 heartbeater.cc:511] Master 127.19.202.126:38209 requested a full tablet report, sending...
I20260810 22:43:08.584770 8358 ts_manager.cc:194] Registered new tserver with Master: c4029ed684a346d9881a71a2fb9bd935 (127.19.202.67:45307)
I20260810 22:43:08.585332 8358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.67:40599
I20260810 22:43:08.588403 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:43:08.594002 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.590391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.19.202.126:38209)
0810 22:43:08.590521 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.590525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.590704 (+ 179us) server_negotiation.cc:407] Connection header received
0810 22:43:08.590796 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.590798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.590862 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.590928 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:08.591181 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.591662 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.592508 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.592706 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.592835 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.593573 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:08.593576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:08.593577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:08.593588 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:08.593618 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:08.593625 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:08.593721 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.593811 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.593840 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:43:08.596589 8358 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:43:08.599646 8358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:43:08.613387 8903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.607878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39265 (local address 127.0.0.1:44986)
0810 22:43:08.608244 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.608294 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.608427 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.609247 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.609253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.609295 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:08.609474 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.609483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.610727 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.610732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.611790 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.611801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.612853 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.612856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.612966 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.613007 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.613121 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.613230 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260810 22:43:08.613421 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.608042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.19.202.66:39265)
0810 22:43:08.608552 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.608566 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.608594 (+ 28us) server_negotiation.cc:407] Connection header received
0810 22:43:08.608675 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.608680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.608905 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.609064 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:08.609614 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.610616 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.611930 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.612696 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.612817 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.613033 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.613163 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.613257 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":137,"threads_started":1}
I20260810 22:43:08.614204 8906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.608127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41935 (local address 127.0.0.1:49734)
0810 22:43:08.608908 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.608936 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.609026 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.609650 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.609653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.609670 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:08.609759 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.609764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.611364 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.611368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.612549 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.612559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.613693 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.613696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.613769 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.613886 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.613958 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.614048 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":142,"thread_start_us":89,"threads_started":1}
I20260810 22:43:08.614415 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.608683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.19.202.65:41935)
0810 22:43:08.609155 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.609167 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.609187 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:43:08.609285 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.609289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.609490 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.609918 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:08.610067 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.611242 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.612689 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.613539 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.613867 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.614085 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.614131 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.614216 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":136,"threads_started":1}
I20260810 22:43:08.614425 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.608807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.19.202.67:45307)
0810 22:43:08.609564 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.609578 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.609600 (+ 22us) server_negotiation.cc:407] Connection header received
0810 22:43:08.609673 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.609678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.609851 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.610039 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:08.610272 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.611353 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.612669 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.613521 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.613639 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.613815 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.614132 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.614216 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":170,"threads_started":1}
I20260810 22:43:08.615168 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.608606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45307 (local address 127.0.0.1:48102)
0810 22:43:08.609200 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.609213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.609316 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.610046 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.610049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.610069 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:08.610151 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.610157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.611466 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.611471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.612526 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.612538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.613657 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.613664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.613767 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.613858 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.613929 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.613996 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":68,"spinlock_wait_cycles":1280,"thread_start_us":91,"threads_started":1}
I20260810 22:43:08.617038 8506 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf5c47de79f4f269a5e1f8558c79496 (DEFAULT_TABLE table=test-table [id=41a25fea9cb347ba94ef1ac335fcbfcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:08.617077 8665 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf5c47de79f4f269a5e1f8558c79496 (DEFAULT_TABLE table=test-table [id=41a25fea9cb347ba94ef1ac335fcbfcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:08.618047 8506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf5c47de79f4f269a5e1f8558c79496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:08.618121 8665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf5c47de79f4f269a5e1f8558c79496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:08.617766 8816 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf5c47de79f4f269a5e1f8558c79496 (DEFAULT_TABLE table=test-table [id=41a25fea9cb347ba94ef1ac335fcbfcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:08.618659 8816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf5c47de79f4f269a5e1f8558c79496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:08.624384 8912 tablet_bootstrap.cc:492] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479: Bootstrap starting.
I20260810 22:43:08.625926 8915 tablet_bootstrap.cc:492] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Bootstrap starting.
I20260810 22:43:08.627372 8912 tablet_bootstrap.cc:654] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:08.627741 8915 tablet_bootstrap.cc:654] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:08.628159 8912 log.cc:824] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:08.628430 8915 log.cc:824] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:08.628919 8913 tablet_bootstrap.cc:492] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Bootstrap starting.
I20260810 22:43:08.630424 8915 tablet_bootstrap.cc:492] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: No bootstrap required, opened a new log
I20260810 22:43:08.630661 8915 ts_tablet_manager.cc:1397] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 22:43:08.630976 8912 tablet_bootstrap.cc:492] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479: No bootstrap required, opened a new log
I20260810 22:43:08.631074 8913 tablet_bootstrap.cc:654] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:08.631199 8912 ts_tablet_manager.cc:1397] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 22:43:08.631758 8913 log.cc:824] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:08.633852 8913 tablet_bootstrap.cc:492] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: No bootstrap required, opened a new log
I20260810 22:43:08.634050 8913 ts_tablet_manager.cc:1397] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:43:08.635083 8915 raft_consensus.cc:348] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.635083 8912 raft_consensus.cc:348] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.635370 8912 raft_consensus.cc:374] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:08.635381 8915 raft_consensus.cc:374] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:08.635427 8915 raft_consensus.cc:729] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4029ed684a346d9881a71a2fb9bd935, State: Initialized, Role: FOLLOWER
I20260810 22:43:08.635435 8912 raft_consensus.cc:729] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d279a43b11974ab6a24525ca714ec479, State: Initialized, Role: FOLLOWER
I20260810 22:43:08.635900 8915 consensus_queue.cc:260] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.635901 8912 consensus_queue.cc:260] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.636968 8915 ts_tablet_manager.cc:1428] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Time spent starting tablet: real 0.006s user 0.003s sys 0.005s
I20260810 22:43:08.636968 8912 ts_tablet_manager.cc:1428] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260810 22:43:08.637352 8891 heartbeater.cc:503] Master 127.19.202.126:38209 was elected leader, sending a full tablet report...
I20260810 22:43:08.637470 8589 heartbeater.cc:503] Master 127.19.202.126:38209 was elected leader, sending a full tablet report...
I20260810 22:43:08.637619 8913 raft_consensus.cc:348] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.637848 8913 raft_consensus.cc:374] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:08.637910 8913 raft_consensus.cc:729] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c9ee8463de74c49adc87ab85f09ce38, State: Initialized, Role: FOLLOWER
I20260810 22:43:08.638350 8913 consensus_queue.cc:260] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.639190 8913 ts_tablet_manager.cc:1428] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:43:08.639533 8742 heartbeater.cc:503] Master 127.19.202.126:38209 was elected leader, sending a full tablet report...
W20260810 22:43:08.643044 8743 tablet.cc:2369] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:43:08.694686 8590 tablet.cc:2369] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:43:08.822911 8893 tablet.cc:2369] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:43:08.900873 8920 raft_consensus.cc:482] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:08.901188 8920 raft_consensus.cc:504] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.902340 8920 leader_election.cc:288] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c9ee8463de74c49adc87ab85f09ce38 (127.19.202.66:39265), c4029ed684a346d9881a71a2fb9bd935 (127.19.202.67:45307)
I20260810 22:43:08.906888 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.902644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:39265 (local address 127.19.202.65:48181)
0810 22:43:08.903024 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.903044 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.903164 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.903480 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.903483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.903503 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:08.903638 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.903645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.904712 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.904715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.905649 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.905658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.906444 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.906448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.906536 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.906565 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.906653 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.906724 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
I20260810 22:43:08.906913 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.902746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:48181 (local address 127.19.202.66:39265)
0810 22:43:08.902890 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.902894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.903133 (+ 239us) server_negotiation.cc:407] Connection header received
0810 22:43:08.903287 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.903290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.903351 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.903444 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:08.903817 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.904579 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.905780 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.906315 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.906416 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.906456 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.906699 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.906774 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:43:08.907184 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.903259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:35181 (local address 127.19.202.67:45307)
0810 22:43:08.903389 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.903395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.903538 (+ 143us) server_negotiation.cc:407] Connection header received
0810 22:43:08.903750 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.903760 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.903837 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.903918 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:08.904243 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.904992 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.906101 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.906588 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.906700 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.906741 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.906978 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.907079 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 22:43:08.907156 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.903129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:45307 (local address 127.19.202.65:35181)
0810 22:43:08.903452 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:08.903466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:08.903618 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:08.903960 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:08.903963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:08.903982 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:08.904104 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.904116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.905113 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.905116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.905950 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:08.905979 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.906723 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.906727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:08.906818 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.906841 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 22:43:08.906931 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 22:43:08.907007 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":71,"threads_started":1}
I20260810 22:43:08.908067 8839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf5c47de79f4f269a5e1f8558c79496" candidate_uuid: "d279a43b11974ab6a24525ca714ec479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4029ed684a346d9881a71a2fb9bd935" is_pre_election: true
I20260810 22:43:08.908070 8688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf5c47de79f4f269a5e1f8558c79496" candidate_uuid: "d279a43b11974ab6a24525ca714ec479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c9ee8463de74c49adc87ab85f09ce38" is_pre_election: true
I20260810 22:43:08.908377 8688 raft_consensus.cc:2454] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d279a43b11974ab6a24525ca714ec479 in term 0.
I20260810 22:43:08.908386 8839 raft_consensus.cc:2454] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d279a43b11974ab6a24525ca714ec479 in term 0.
I20260810 22:43:08.908900 8452 leader_election.cc:302] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4029ed684a346d9881a71a2fb9bd935, d279a43b11974ab6a24525ca714ec479; no voters:
I20260810 22:43:08.909191 8920 raft_consensus.cc:2781] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:43:08.909292 8920 raft_consensus.cc:482] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:43:08.909350 8920 raft_consensus.cc:3037] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:08.910408 8920 raft_consensus.cc:504] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.910795 8920 leader_election.cc:288] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [CANDIDATE]: Term 1 election: Requested vote from peers 3c9ee8463de74c49adc87ab85f09ce38 (127.19.202.66:39265), c4029ed684a346d9881a71a2fb9bd935 (127.19.202.67:45307)
I20260810 22:43:08.911235 8688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf5c47de79f4f269a5e1f8558c79496" candidate_uuid: "d279a43b11974ab6a24525ca714ec479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c9ee8463de74c49adc87ab85f09ce38"
I20260810 22:43:08.911379 8688 raft_consensus.cc:3037] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:08.912405 8688 raft_consensus.cc:2454] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d279a43b11974ab6a24525ca714ec479 in term 1.
I20260810 22:43:08.912698 8839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf5c47de79f4f269a5e1f8558c79496" candidate_uuid: "d279a43b11974ab6a24525ca714ec479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4029ed684a346d9881a71a2fb9bd935"
I20260810 22:43:08.912739 8452 leader_election.cc:302] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c9ee8463de74c49adc87ab85f09ce38, d279a43b11974ab6a24525ca714ec479; no voters:
I20260810 22:43:08.912891 8839 raft_consensus.cc:3037] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:08.912961 8920 raft_consensus.cc:2781] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:08.913195 8920 raft_consensus.cc:686] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [term 1 LEADER]: Becoming Leader. State: Replica: d279a43b11974ab6a24525ca714ec479, State: Running, Role: LEADER
I20260810 22:43:08.913852 8920 consensus_queue.cc:237] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } }
I20260810 22:43:08.914192 8839 raft_consensus.cc:2454] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d279a43b11974ab6a24525ca714ec479 in term 1.
I20260810 22:43:08.917303 8357 catalog_manager.cc:5495] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 reported cstate change: term changed from 0 to 1, leader changed from <none> to d279a43b11974ab6a24525ca714ec479 (127.19.202.65). New cstate: current_term: 1 leader_uuid: "d279a43b11974ab6a24525ca714ec479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d279a43b11974ab6a24525ca714ec479" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 41935 } health_report { overall_health: HEALTHY } } }
I20260810 22:43:08.983410 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:08.980133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.19.202.65:41935)
0810 22:43:08.980270 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:08.980273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:08.980287 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:43:08.980363 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:08.980365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:08.980430 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:08.980534 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:08.980790 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.981290 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.982127 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:08.982328 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:08.982414 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:08.982666 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:08.983052 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:08.983160 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:08.983258 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 22:43:08.983290 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:43:08.989632 8839 raft_consensus.cc:1261] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 1 FOLLOWER]: Refusing update from remote peer d279a43b11974ab6a24525ca714ec479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:08.989632 8688 raft_consensus.cc:1261] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 1 FOLLOWER]: Refusing update from remote peer d279a43b11974ab6a24525ca714ec479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:08.990470 8925 consensus_queue.cc:1035] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4029ed684a346d9881a71a2fb9bd935" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 45307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:08.990725 8920 consensus_queue.cc:1035] T 2cf5c47de79f4f269a5e1f8558c79496 P d279a43b11974ab6a24525ca714ec479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c9ee8463de74c49adc87ab85f09ce38" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:09.004719 8929 mvcc.cc:205] Tried to move back new op lower bound from 7317101727692513280 to 7317101727394422784. Current Snapshot: MvccSnapshot[applied={T|T < 7317101727692513280 or (T in {7317101727692513280})}]
I20260810 22:43:09.008659 8356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:09.008845 8356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:09.011164 8356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22188e37472044a7b765982c959cacf4: Sending DeleteTablet for 3 replicas of tablet 2cf5c47de79f4f269a5e1f8558c79496
I20260810 22:43:09.012053 8816 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf5c47de79f4f269a5e1f8558c79496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:09 UTC) from {username='slave'} at 127.0.0.1:48102
I20260810 22:43:09.012101 8665 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf5c47de79f4f269a5e1f8558c79496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:09 UTC) from {username='slave'} at 127.0.0.1:44986
I20260810 22:43:09.012557 8940 tablet_replica.cc:330] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: stopping tablet replica
I20260810 22:43:09.012563 8939 tablet_replica.cc:330] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: stopping tablet replica
I20260810 22:43:09.012562 8506 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf5c47de79f4f269a5e1f8558c79496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:09 UTC) from {username='slave'} at 127.0.0.1:49734
I20260810 22:43:09.012916 8939 raft_consensus.cc:2229] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:09.012916 8940 raft_consensus.cc:2229] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:09.013164 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 8430
I20260810 22:43:09.013232 8940 raft_consensus.cc:2258] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:09.013257 8939 raft_consensus.cc:2258] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:09.016652 8939 ts_tablet_manager.cc:1905] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:09.017040 8940 ts_tablet_manager.cc:1905] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:09.019548 8939 ts_tablet_manager.cc:1918] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:09.019663 8939 log.cc:1196] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/2cf5c47de79f4f269a5e1f8558c79496
I20260810 22:43:09.019742 8940 ts_tablet_manager.cc:1918] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:09.019841 8940 log.cc:1196] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/2cf5c47de79f4f269a5e1f8558c79496
I20260810 22:43:09.020005 8939 ts_tablet_manager.cc:1939] T 2cf5c47de79f4f269a5e1f8558c79496 P c4029ed684a346d9881a71a2fb9bd935: Deleting consensus metadata
I20260810 22:43:09.020179 8940 ts_tablet_manager.cc:1939] T 2cf5c47de79f4f269a5e1f8558c79496 P 3c9ee8463de74c49adc87ab85f09ce38: Deleting consensus metadata
I20260810 22:43:09.021291 8344 catalog_manager.cc:4841] TS c4029ed684a346d9881a71a2fb9bd935 (127.19.202.67:45307): tablet 2cf5c47de79f4f269a5e1f8558c79496 (table test-table [id=41a25fea9cb347ba94ef1ac335fcbfcc]) successfully deleted
I20260810 22:43:09.021514 8344 catalog_manager.cc:4841] TS 3c9ee8463de74c49adc87ab85f09ce38 (127.19.202.66:39265): tablet 2cf5c47de79f4f269a5e1f8558c79496 (table test-table [id=41a25fea9cb347ba94ef1ac335fcbfcc]) successfully deleted
W20260810 22:43:09.023043 8340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:41935 (error 108)
I20260810 22:43:09.023758 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 8592
I20260810 22:43:09.024076 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:09.023770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:41935 (local address 127.0.0.1:49758)
0810 22:43:09.023918 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:09.023988 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 22:43:09.025398 8340 catalog_manager.cc:4584] TS d279a43b11974ab6a24525ca714ec479 (127.19.202.65:41935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2cf5c47de79f4f269a5e1f8558c79496: Network error: Client connection negotiation failed: client connection to 127.19.202.65:41935: connect: Connection refused (error 111)
I20260810 22:43:09.031728 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 8745
I20260810 22:43:09.038205 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 8322
2026-08-10T22:43:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1298 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:09 dist-test-slave-1x4g krb5kdc[8946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:09 dist-test-slave-1x4g krb5kdc[8946](info): set up 2 sockets
Aug 10 22:43:09 dist-test-slave-1x4g krb5kdc[8946](info): commencing operation
krb5kdc: starting...
W20260810 22:43:10.129091 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:10 dist-test-slave-1x4g krb5kdc[8946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:10.252213 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:40385
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:40385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:10.365832 8963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:10.366117 8963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:10.366194 8963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:10.370112 8963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:10.370199 8963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:10.370239 8963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:10.370292 8963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:10.370332 8963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:10.375715 8963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:40385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:40385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:10.376889 8963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:10.377969 8963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:10.386300 8970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:10.386391 8963 server_base.cc:1034] running on GCE node
W20260810 22:43:10.386308 8969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:10.386317 8973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:10.387131 8963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:10.388311 8963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:10.389505 8963 hybrid_clock.cc:648] HybridClock initialized: now 1786401790389475 us; error 40 us; skew 500 ppm
Aug 10 22:43:10 dist-test-slave-1x4g krb5kdc[8946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401790, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:10.392422 8963 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:10.393381 8963 webserver.cc:460] Webserver started at http://127.19.202.126:43985/ using document root <none> and password file <none>
I20260810 22:43:10.393941 8963 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:10.394001 8963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:10.394236 8963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:10.396306 8963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "8576eadb2f334d4ab73989284b28ec2d"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "85dd537ac2a8e38a620595fa7f29c956"
server_key_iv: "977a3a2a6f4caa45d7a7207800803e3f"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.396960 8963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "8576eadb2f334d4ab73989284b28ec2d"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "85dd537ac2a8e38a620595fa7f29c956"
server_key_iv: "977a3a2a6f4caa45d7a7207800803e3f"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.400674 8963 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:43:10.403228 8980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:10.404470 8963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:10.404619 8963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "8576eadb2f334d4ab73989284b28ec2d"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "85dd537ac2a8e38a620595fa7f29c956"
server_key_iv: "977a3a2a6f4caa45d7a7207800803e3f"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.404733 8963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:10.425987 8963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:10.429075 8963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:10.429286 8963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:10.436612 8963 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:40385
I20260810 22:43:10.436643 9045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:40385 every 8 connection(s)
I20260810 22:43:10.437651 8963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:10.438908 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 8963
I20260810 22:43:10.439051 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:10.439327 20265 external_mini_cluster.cc:1428] Setting key aff77950e882c9a0482fbfd05503e37c
I20260810 22:43:10.440752 9046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:10.447571 9046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d: Bootstrap starting.
Aug 10 22:43:10 dist-test-slave-1x4g krb5kdc[8946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:10.450081 9046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:10.451037 9046 log.cc:824] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:10.453083 9046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d: No bootstrap required, opened a new log
I20260810 22:43:10.455142 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:10.440745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.19.202.126:40385)
0810 22:43:10.441268 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:10.441288 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:10.441330 (+ 42us) server_negotiation.cc:407] Connection header received
0810 22:43:10.442175 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:10.442199 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:10.442620 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:10.442974 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:10.443635 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.445143 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.445990 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.446757 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.446942 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:10.449468 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:10.449474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:10.449490 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:10.449533 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:10.451546 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:10.452000 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:10.452005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:10.452009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:10.452120 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:10.452423 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:10.452430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:10.452436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:10.452893 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:10.453098 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:10.453799 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 22:43:10.454170 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":147,"threads_started":1}
I20260810 22:43:10.456511 9046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8576eadb2f334d4ab73989284b28ec2d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40385 } }
I20260810 22:43:10.456825 9046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:10.456911 9046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8576eadb2f334d4ab73989284b28ec2d, State: Initialized, Role: FOLLOWER
I20260810 22:43:10.457379 9046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8576eadb2f334d4ab73989284b28ec2d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40385 } }
I20260810 22:43:10.457530 9046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:10.457612 9046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:10.457713 9046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:10.458626 9046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8576eadb2f334d4ab73989284b28ec2d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40385 } }
I20260810 22:43:10.458976 9046 leader_election.cc:302] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8576eadb2f334d4ab73989284b28ec2d; no voters:
I20260810 22:43:10.459301 9046 leader_election.cc:288] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:10.459432 9051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:10.459650 9051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [term 1 LEADER]: Becoming Leader. State: Replica: 8576eadb2f334d4ab73989284b28ec2d, State: Running, Role: LEADER
I20260810 22:43:10.460016 9051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8576eadb2f334d4ab73989284b28ec2d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40385 } }
I20260810 22:43:10.460489 9046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:10.461645 9052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8576eadb2f334d4ab73989284b28ec2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8576eadb2f334d4ab73989284b28ec2d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40385 } } }
I20260810 22:43:10.461783 9052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:10.461935 9053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8576eadb2f334d4ab73989284b28ec2d. Latest consensus state: current_term: 1 leader_uuid: "8576eadb2f334d4ab73989284b28ec2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8576eadb2f334d4ab73989284b28ec2d" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40385 } } }
I20260810 22:43:10.462090 9053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:10.462394 9060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:10.465588 9060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:10.470747 9060 catalog_manager.cc:1347] Generated new cluster ID: 9cbe1445aa754de2be96b44d83c01482
I20260810 22:43:10.470814 9060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:10.479393 9060 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:10.480225 9060 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:10.486183 9060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d: Generated new TSK 0
I20260810 22:43:10.486799 9060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.202.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
Entry for principal kudu/127.19.202.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab.
I20260810 22:43:10.523281 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:40385
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:10.659585 9074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:10.659857 9074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:10.659904 9074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:10.663903 9074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:10.663961 9074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:10.664018 9074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:43:10.669291 9074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.65.keytab
--principal=kudu/127.19.202.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:40385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:10.670279 9074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:10.671259 9074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:10.681109 9081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:10.681115 9083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:10.681392 9074 server_base.cc:1034] running on GCE node
W20260810 22:43:10.681437 9080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:10.681998 9074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:10.682655 9074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:10.683851 9074 hybrid_clock.cc:648] HybridClock initialized: now 1786401790683842 us; error 46 us; skew 500 ppm
Aug 10 22:43:10 dist-test-slave-1x4g krb5kdc[8946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401790, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:10.686877 9074 init.cc:377] Logged in from keytab as kudu/127.19.202.65@KRBTEST.COM (short username kudu)
I20260810 22:43:10.687736 9074 webserver.cc:460] Webserver started at http://127.19.202.65:46581/ using document root <none> and password file <none>
I20260810 22:43:10.688263 9074 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:10.688335 9074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:10.688516 9074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:10.690639 9074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "c83ad6b836f1468aadbd77af5f3c264b"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "14080ea3fbdf038c7d5547f8c1615504"
server_key_iv: "5486a3b9e96abae6720836696032bea3"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.691180 9074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "c83ad6b836f1468aadbd77af5f3c264b"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "14080ea3fbdf038c7d5547f8c1615504"
server_key_iv: "5486a3b9e96abae6720836696032bea3"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.695062 9074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 22:43:10.697615 9091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:10.698855 9074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:10.698951 9074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "c83ad6b836f1468aadbd77af5f3c264b"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "14080ea3fbdf038c7d5547f8c1615504"
server_key_iv: "5486a3b9e96abae6720836696032bea3"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.699038 9074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:10.713932 9074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:10.717260 9074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:10.717420 9074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:10.717969 9074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:10.718740 9074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:10.718794 9074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:10.718834 9074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:10.718858 9074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:10.727744 9074 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:34907
I20260810 22:43:10.727779 9218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:34907 every 8 connection(s)
I20260810 22:43:10.728612 9074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:43:10.733457 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 9074
I20260810 22:43:10.733606 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:43:10.733922 20265 external_mini_cluster.cc:1428] Setting key 3e222489d1f529a6577f6dd2eb4b7f2e
Aug 10 22:43:10 dist-test-slave-1x4g krb5kdc[8946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401790, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.65@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:10.743489 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:10.730558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:45635 (local address 127.19.202.126:40385)
0810 22:43:10.730722 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:10.730727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:10.731524 (+ 797us) server_negotiation.cc:407] Connection header received
0810 22:43:10.732826 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:10.732830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:10.732900 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:10.732978 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:10.734231 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.734889 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.736499 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.736765 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.736897 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:10.739495 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:10.739499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:10.739502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:10.739532 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:10.741226 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:10.741827 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:10.741832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:10.741834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:10.741929 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:10.742315 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:10.742319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:10.742321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:10.742515 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:10.742635 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:10.743163 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 22:43:10.743296 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.19.202.66@KRBTEST.COM; defaulting to no policy
I20260810 22:43:10.744345 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:10.730933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:40385 (local address 127.19.202.65:45635)
0810 22:43:10.731362 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:10.731405 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:10.732601 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:10.733128 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:10.733134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:10.733468 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:10.734085 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:10.734102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.735014 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.735018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:10.736374 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:10.736390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.736951 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.736957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:10.737188 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:10.737856 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:10.737873 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:10.739323 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:10.741391 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:10.741397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:10.741412 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:10.741700 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:10.742036 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:10.742040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:10.742042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:10.742200 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:10.742653 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:10.742662 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:10.742912 (+ 250us) client_negotiation.cc:769] Sending connection context
0810 22:43:10.743104 (+ 192us) client_negotiation.cc:240] Negotiation successful
0810 22:43:10.743449 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":109,"threads_started":1}
I20260810 22:43:10.745702 9220 heartbeater.cc:347] Connected to a master server at 127.19.202.126:40385
I20260810 22:43:10.745949 9220 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:10.746536 9220 heartbeater.cc:511] Master 127.19.202.126:40385 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.66@KRBTEST.COM" created.
I20260810 22:43:10.748162 9000 ts_manager.cc:194] Registered new tserver with Master: c83ad6b836f1468aadbd77af5f3c264b (127.19.202.65:34907)
I20260810 22:43:10.749483 9000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.65@KRBTEST.COM'} at 127.19.202.65:45635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
Entry for principal kudu/127.19.202.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab.
I20260810 22:43:10.763957 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:40385
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:10.880019 9226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:10.880270 9226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:10.880311 9226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:10.884163 9226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:10.884212 9226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:10.884264 9226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:43:10.889341 9226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.66.keytab
--principal=kudu/127.19.202.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:40385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:10.890321 9226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:10.891258 9226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:10.900658 9232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:10.900658 9233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:10.900875 9235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:10.901033 9226 server_base.cc:1034] running on GCE node
I20260810 22:43:10.901458 9226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:10.902150 9226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:10.903326 9226 hybrid_clock.cc:648] HybridClock initialized: now 1786401790903295 us; error 41 us; skew 500 ppm
Aug 10 22:43:10 dist-test-slave-1x4g krb5kdc[8946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401790, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:10.906132 9226 init.cc:377] Logged in from keytab as kudu/127.19.202.66@KRBTEST.COM (short username kudu)
I20260810 22:43:10.906991 9226 webserver.cc:460] Webserver started at http://127.19.202.66:44249/ using document root <none> and password file <none>
I20260810 22:43:10.907586 9226 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:10.907671 9226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:10.907895 9226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:10.910159 9226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "86aa678410124422bb0492f70022acae"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "561243e13cbc6df39a960e9fbc858236"
server_key_iv: "cca7045a1a21d55bd68a5ad83729f06d"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.910719 9226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "86aa678410124422bb0492f70022acae"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "561243e13cbc6df39a960e9fbc858236"
server_key_iv: "cca7045a1a21d55bd68a5ad83729f06d"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.914477 9226 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:43:10.916941 9243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:10.918119 9226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:10.918246 9226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "86aa678410124422bb0492f70022acae"
format_stamp: "Formatted at 2026-08-10 22:43:10 on dist-test-slave-1x4g"
server_key: "561243e13cbc6df39a960e9fbc858236"
server_key_iv: "cca7045a1a21d55bd68a5ad83729f06d"
server_key_version: "encryptionkey@0"
I20260810 22:43:10.918351 9226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:10.943405 9226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:10.946873 9226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:10.947104 9226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:10.947702 9226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:10.948573 9226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:10.948639 9226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:10.948706 9226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:10.948751 9226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:10.957792 9226 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:33805
I20260810 22:43:10.957818 9372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:33805 every 8 connection(s)
I20260810 22:43:10.958709 9226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:43:10.961138 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 9226
I20260810 22:43:10.961290 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:43:10.961660 20265 external_mini_cluster.cc:1428] Setting key 7c3869cb169647d9b0bc24b596afa81c
Aug 10 22:43:10 dist-test-slave-1x4g krb5kdc[8946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401790, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.66@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.202.67@KRBTEST.COM; defaulting to no policy
I20260810 22:43:10.974117 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:10.960514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:56369 (local address 127.19.202.126:40385)
0810 22:43:10.960672 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:10.960678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:10.961667 (+ 989us) server_negotiation.cc:407] Connection header received
0810 22:43:10.963430 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:10.963434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:10.963485 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:10.963594 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:10.964745 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.965567 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.967170 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.967388 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.967493 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:10.970518 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:10.970521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:10.970523 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:10.970547 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:10.971989 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:10.972618 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:10.972621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:10.972623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:10.972707 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:10.973073 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:10.973075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:10.973077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:10.973259 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:10.973363 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:10.973887 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 22:43:10.973994 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.67@KRBTEST.COM" created.
I20260810 22:43:10.975067 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:10.960945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:40385 (local address 127.19.202.66:56369)
0810 22:43:10.961456 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:10.961510 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:10.963205 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:10.963768 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:10.963775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:10.964139 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:10.964606 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:10.964619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.965694 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.965699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:10.967050 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:10.967060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:10.967543 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:10.967547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:10.967711 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:10.968280 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:10.968296 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:10.970340 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:10.972110 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:10.972121 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:10.972140 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:10.972502 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:10.972829 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:10.972832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:10.972834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:10.972989 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:10.973353 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:10.973358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:10.973630 (+ 272us) client_negotiation.cc:769] Sending connection context
0810 22:43:10.973811 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 22:43:10.974093 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":129,"threads_started":1}
I20260810 22:43:10.976240 9373 heartbeater.cc:347] Connected to a master server at 127.19.202.126:40385
I20260810 22:43:10.976511 9373 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:10.977209 9373 heartbeater.cc:511] Master 127.19.202.126:40385 requested a full tablet report, sending...
I20260810 22:43:10.978394 9000 ts_manager.cc:194] Registered new tserver with Master: 86aa678410124422bb0492f70022acae (127.19.202.66:33805)
I20260810 22:43:10.979009 9000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.66@KRBTEST.COM'} at 127.19.202.66:56369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
Entry for principal kudu/127.19.202.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab.
I20260810 22:43:10.991539 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:40385
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:11.103509 9378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:11.103765 9378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:11.103829 9378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:11.107836 9378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:11.107890 9378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:11.107969 9378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:43:11.113276 9378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.67.keytab
--principal=kudu/127.19.202.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:40385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:11.114269 9378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:11.115212 9378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:11.124939 9386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:11.124936 9384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:11.124950 9383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:11.125296 9378 server_base.cc:1034] running on GCE node
I20260810 22:43:11.125674 9378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:11.126252 9378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:11.127408 9378 hybrid_clock.cc:648] HybridClock initialized: now 1786401791127396 us; error 29 us; skew 500 ppm
Aug 10 22:43:11 dist-test-slave-1x4g krb5kdc[8946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401791, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:11.130182 9378 init.cc:377] Logged in from keytab as kudu/127.19.202.67@KRBTEST.COM (short username kudu)
I20260810 22:43:11.131023 9378 webserver.cc:460] Webserver started at http://127.19.202.67:41439/ using document root <none> and password file <none>
I20260810 22:43:11.131537 9378 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:11.131594 9378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:11.131752 9378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:11.133899 9378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "c7377191f43e4b7482fda868ecd734a2"
format_stamp: "Formatted at 2026-08-10 22:43:11 on dist-test-slave-1x4g"
server_key: "274e44ddd5a5bc5559c73446cdc52508"
server_key_iv: "8166507d1703d1076322b97be1652be6"
server_key_version: "encryptionkey@0"
I20260810 22:43:11.134421 9378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "c7377191f43e4b7482fda868ecd734a2"
format_stamp: "Formatted at 2026-08-10 22:43:11 on dist-test-slave-1x4g"
server_key: "274e44ddd5a5bc5559c73446cdc52508"
server_key_iv: "8166507d1703d1076322b97be1652be6"
server_key_version: "encryptionkey@0"
I20260810 22:43:11.138077 9378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:11.140399 9394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:11.141630 9378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:11.141723 9378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "c7377191f43e4b7482fda868ecd734a2"
format_stamp: "Formatted at 2026-08-10 22:43:11 on dist-test-slave-1x4g"
server_key: "274e44ddd5a5bc5559c73446cdc52508"
server_key_iv: "8166507d1703d1076322b97be1652be6"
server_key_version: "encryptionkey@0"
I20260810 22:43:11.141805 9378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:11.154337 9378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:11.157003 9378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:11.157167 9378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:11.157673 9378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:11.158521 9378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:11.158567 9378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:11.158604 9378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:11.158627 9378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:11.167493 9378 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:35555
I20260810 22:43:11.167526 9531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:35555 every 8 connection(s)
I20260810 22:43:11.168404 9378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
Aug 10 22:43:11 dist-test-slave-1x4g krb5kdc[8946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401791, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.67@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:11.178200 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 9378
I20260810 22:43:11.178351 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:43:11.178670 20265 external_mini_cluster.cc:1428] Setting key 0d646ef7ff8f967f73ed1e6ce7ef0f22
I20260810 22:43:11.181993 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.170160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:45639 (local address 127.19.202.126:40385)
0810 22:43:11.170303 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.170306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.171099 (+ 793us) server_negotiation.cc:407] Connection header received
0810 22:43:11.172481 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.172484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.172539 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.172645 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:11.173840 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.174381 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.175582 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.175794 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.175906 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.178321 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:11.178324 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:11.178326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:11.178349 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:11.180037 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:11.180557 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:11.180561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:11.180562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:11.180657 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:11.181062 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:11.181064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:11.181065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:11.181237 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:11.181329 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.181771 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.181865 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 22:43:11.182773 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.170524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:40385 (local address 127.19.202.67:45639)
0810 22:43:11.170957 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.171001 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.172263 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.172875 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.172882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.173224 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:11.173698 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.173711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.174503 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.174507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.175455 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.175465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.175940 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.175944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.176111 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.176670 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:11.176686 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:11.178160 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:11.180185 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:11.180189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:11.180200 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:11.180463 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:11.180762 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:11.180764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:11.180766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:11.180977 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:11.181340 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:11.181345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:11.181569 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.181713 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.181978 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":122,"threads_started":1}
I20260810 22:43:11.183804 9532 heartbeater.cc:347] Connected to a master server at 127.19.202.126:40385
I20260810 22:43:11.184039 9532 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:11.184522 9532 heartbeater.cc:511] Master 127.19.202.126:40385 requested a full tablet report, sending...
I20260810 22:43:11.185497 9000 ts_manager.cc:194] Registered new tserver with Master: c7377191f43e4b7482fda868ecd734a2 (127.19.202.67:35555)
I20260810 22:43:11.186045 9000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.202.67@KRBTEST.COM'} at 127.19.202.67:45639
I20260810 22:43:11.192253 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:43:11.201367 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.194245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.19.202.126:40385)
0810 22:43:11.194379 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.194383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.194571 (+ 188us) server_negotiation.cc:407] Connection header received
0810 22:43:11.194682 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.194684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.194721 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.194776 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:11.195406 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.195867 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.196732 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.197013 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.197119 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.198494 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:11.198498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:11.198501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:11.198546 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:11.199670 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:11.200087 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:11.200090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:11.200110 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:11.200197 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:11.200468 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:11.200471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:11.200473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:11.200683 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:11.200854 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.201083 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.201194 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:43:11.203987 9000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:43:11.206964 9000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:43:11.221611 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.216033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42104 (local address 127.19.202.67:35555)
0810 22:43:11.216743 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.216758 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.216784 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:43:11.216900 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.216906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.217088 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.217238 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:11.217495 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.218283 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.220140 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.220928 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.221043 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.221181 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.221365 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.221460 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":92,"threads_started":1}
I20260810 22:43:11.221602 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.215264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:35555 (local address 127.0.0.1:42104)
0810 22:43:11.216264 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.216277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.216366 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.217214 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.217216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.217234 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:11.217340 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.217350 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.218405 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.218412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.220002 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.220011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.221043 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.221047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.221153 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.221193 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.221334 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.221468 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":180,"thread_start_us":83,"threads_started":1}
I20260810 22:43:11.221621 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.215975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:33805 (local address 127.0.0.1:46390)
0810 22:43:11.216548 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.216561 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.216698 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.217446 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.217449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.217469 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:11.217550 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.217555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.218822 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.218825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.219761 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.219773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.221108 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.221111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.221202 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.221226 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.221321 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.221475 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":74,"thread_start_us":87,"threads_started":1}
I20260810 22:43:11.222153 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.216146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.19.202.66:33805)
0810 22:43:11.217014 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.217029 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.217054 (+ 25us) server_negotiation.cc:407] Connection header received
0810 22:43:11.217130 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.217134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.217302 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.217455 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:11.217663 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.218696 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.219899 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.220724 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.220887 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.221048 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.221911 (+ 863us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.222020 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":126,"threads_started":1}
I20260810 22:43:11.222342 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.215228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:34907 (local address 127.0.0.1:46336)
0810 22:43:11.215666 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.215705 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.215806 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.216712 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.216719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.216744 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:11.216949 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.216957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.218602 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.218605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.220031 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.220040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.221905 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.221911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.222015 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.222037 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.222107 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.222175 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":148,"thread_start_us":86,"threads_started":1}
I20260810 22:43:11.223088 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.215364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46336 (local address 127.19.202.65:34907)
0810 22:43:11.216099 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.216118 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.216152 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:43:11.216239 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.216247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.216481 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.217116 (+ 635us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:11.217261 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.218470 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.220902 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.221753 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.222640 (+ 887us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.222762 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.222802 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.222899 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":114,"threads_started":1}
I20260810 22:43:11.225343 9146 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa2ee8d77344efe8cbde9a78958847b (DEFAULT_TABLE table=test-table [id=dbf57b149a78437dbf8b6f07992cb58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:11.225322 9298 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa2ee8d77344efe8cbde9a78958847b (DEFAULT_TABLE table=test-table [id=dbf57b149a78437dbf8b6f07992cb58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:11.225322 9456 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa2ee8d77344efe8cbde9a78958847b (DEFAULT_TABLE table=test-table [id=dbf57b149a78437dbf8b6f07992cb58c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:11.226347 9146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa2ee8d77344efe8cbde9a78958847b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:11.226364 9298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa2ee8d77344efe8cbde9a78958847b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:11.226364 9456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa2ee8d77344efe8cbde9a78958847b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:11.232533 9553 tablet_bootstrap.cc:492] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b: Bootstrap starting.
I20260810 22:43:11.233939 9555 tablet_bootstrap.cc:492] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Bootstrap starting.
I20260810 22:43:11.235503 9553 tablet_bootstrap.cc:654] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:11.235533 9555 tablet_bootstrap.cc:654] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:11.236557 9555 log.cc:824] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:11.236557 9553 log.cc:824] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:11.237108 9556 tablet_bootstrap.cc:492] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Bootstrap starting.
I20260810 22:43:11.238837 9553 tablet_bootstrap.cc:492] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b: No bootstrap required, opened a new log
I20260810 22:43:11.239079 9553 ts_tablet_manager.cc:1397] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 22:43:11.239112 9555 tablet_bootstrap.cc:492] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: No bootstrap required, opened a new log
I20260810 22:43:11.239132 9556 tablet_bootstrap.cc:654] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:11.239280 9555 ts_tablet_manager.cc:1397] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 22:43:11.240377 9556 log.cc:824] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:11.242290 9556 tablet_bootstrap.cc:492] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: No bootstrap required, opened a new log
I20260810 22:43:11.242472 9556 ts_tablet_manager.cc:1397] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 22:43:11.242420 9555 raft_consensus.cc:348] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.242691 9555 raft_consensus.cc:374] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:11.242745 9555 raft_consensus.cc:729] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86aa678410124422bb0492f70022acae, State: Initialized, Role: FOLLOWER
I20260810 22:43:11.243083 9553 raft_consensus.cc:348] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.243211 9555 consensus_queue.cc:260] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.243325 9553 raft_consensus.cc:374] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:11.243382 9553 raft_consensus.cc:729] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c83ad6b836f1468aadbd77af5f3c264b, State: Initialized, Role: FOLLOWER
I20260810 22:43:11.243820 9553 consensus_queue.cc:260] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.244215 9555 ts_tablet_manager.cc:1428] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:43:11.244583 9373 heartbeater.cc:503] Master 127.19.202.126:40385 was elected leader, sending a full tablet report...
I20260810 22:43:11.244704 9553 ts_tablet_manager.cc:1428] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:43:11.244897 9220 heartbeater.cc:503] Master 127.19.202.126:40385 was elected leader, sending a full tablet report...
I20260810 22:43:11.246243 9556 raft_consensus.cc:348] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.246500 9556 raft_consensus.cc:374] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:11.246555 9556 raft_consensus.cc:729] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7377191f43e4b7482fda868ecd734a2, State: Initialized, Role: FOLLOWER
I20260810 22:43:11.246961 9556 consensus_queue.cc:260] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.247622 9532 heartbeater.cc:503] Master 127.19.202.126:40385 was elected leader, sending a full tablet report...
I20260810 22:43:11.247887 9556 ts_tablet_manager.cc:1428] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260810 22:43:11.419358 9533 tablet.cc:2369] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:43:11.459873 9374 tablet.cc:2369] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:43:11.479893 9222 tablet.cc:2369] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:43:11.523829 9562 raft_consensus.cc:482] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:11.524029 9562 raft_consensus.cc:504] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.525250 9562 leader_election.cc:288] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c83ad6b836f1468aadbd77af5f3c264b (127.19.202.65:34907), 86aa678410124422bb0492f70022acae (127.19.202.66:33805)
I20260810 22:43:11.526325 9560 raft_consensus.cc:482] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:11.526520 9560 raft_consensus.cc:504] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.528599 9560 leader_election.cc:288] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c83ad6b836f1468aadbd77af5f3c264b (127.19.202.65:34907), c7377191f43e4b7482fda868ecd734a2 (127.19.202.67:35555)
I20260810 22:43:11.529803 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.525532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:34907 (local address 127.19.202.67:38233)
0810 22:43:11.525678 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.525696 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.525817 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.526232 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.526235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.526249 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:11.526378 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.526384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.527243 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.527248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.528439 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.528450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.529373 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.529376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.529482 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.529505 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.529572 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.529646 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":40}
I20260810 22:43:11.530164 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.525962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:57825 (local address 127.19.202.66:33805)
0810 22:43:11.526095 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.526100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.526217 (+ 117us) server_negotiation.cc:407] Connection header received
0810 22:43:11.526327 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.526331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.526388 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.526479 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:11.526826 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.527522 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.528939 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.529490 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.529585 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.529625 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.529945 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.530030 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 22:43:11.530272 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.525654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:38233 (local address 127.19.202.65:34907)
0810 22:43:11.525806 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.525811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.525830 (+ 19us) server_negotiation.cc:407] Connection header received
0810 22:43:11.525957 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.525960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.526028 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.526099 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:11.526514 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.527139 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.528578 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.529166 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.529267 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.529317 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.530061 (+ 744us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.530150 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:43:11.530512 9563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.525806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:33805 (local address 127.19.202.67:57825)
0810 22:43:11.526126 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.526140 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.526231 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.526572 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.526574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.526593 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:11.526707 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.526713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.527646 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.527649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.528814 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.528822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.529710 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.529713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.529797 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.529826 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.530287 (+ 461us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.530362 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":1,"thread_start_us":132,"threads_started":1}
I20260810 22:43:11.530838 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa2ee8d77344efe8cbde9a78958847b" candidate_uuid: "c7377191f43e4b7482fda868ecd734a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83ad6b836f1468aadbd77af5f3c264b" is_pre_election: true
I20260810 22:43:11.531082 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa2ee8d77344efe8cbde9a78958847b" candidate_uuid: "c7377191f43e4b7482fda868ecd734a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86aa678410124422bb0492f70022acae" is_pre_election: true
I20260810 22:43:11.531172 9170 raft_consensus.cc:2454] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7377191f43e4b7482fda868ecd734a2 in term 0.
I20260810 22:43:11.531273 9320 raft_consensus.cc:2454] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7377191f43e4b7482fda868ecd734a2 in term 0.
I20260810 22:43:11.531725 9398 leader_election.cc:302] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86aa678410124422bb0492f70022acae, c7377191f43e4b7482fda868ecd734a2; no voters:
I20260810 22:43:11.532024 9562 raft_consensus.cc:2781] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:43:11.532131 9562 raft_consensus.cc:482] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:43:11.532215 9562 raft_consensus.cc:3037] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:11.532894 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.528317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:49535 (local address 127.19.202.67:35555)
0810 22:43:11.528562 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.528568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.528944 (+ 376us) server_negotiation.cc:407] Connection header received
0810 22:43:11.529119 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.529123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.529166 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.529211 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:11.529592 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.530390 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.531832 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.532389 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.532512 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.532563 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.532723 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.532808 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260810 22:43:11.532924 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.528265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:35555 (local address 127.19.202.66:49535)
0810 22:43:11.528872 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.528884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.529009 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.529317 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.529320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.529334 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:11.529473 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.529483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.530569 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.530576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.531714 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.531723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.532519 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.532523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.532615 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.532645 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.532707 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.532765 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":97,"threads_started":1}
I20260810 22:43:11.533478 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa2ee8d77344efe8cbde9a78958847b" candidate_uuid: "86aa678410124422bb0492f70022acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7377191f43e4b7482fda868ecd734a2" is_pre_election: true
I20260810 22:43:11.533789 9562 raft_consensus.cc:504] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.533980 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.527791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:34907 (local address 127.19.202.66:53035)
0810 22:43:11.528519 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.528533 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:11.528668 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:11.529161 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:11.529164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:11.529179 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:11.529314 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.529320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.530842 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.530845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.532510 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:11.532518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.533498 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.533501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:11.533615 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.533653 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 22:43:11.533772 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 22:43:11.533845 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":63,"threads_started":1}
I20260810 22:43:11.533998 9567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.527846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:53035 (local address 127.19.202.65:34907)
0810 22:43:11.528148 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.528154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.528602 (+ 448us) server_negotiation.cc:407] Connection header received
0810 22:43:11.528890 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.528895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.528947 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.529008 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:11.529446 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.530710 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.532653 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.533233 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.533349 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.533419 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.533792 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.533853 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":77,"threads_started":1}
I20260810 22:43:11.534045 9480 raft_consensus.cc:2379] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86aa678410124422bb0492f70022acae in current term 1: Already voted for candidate c7377191f43e4b7482fda868ecd734a2 in this term.
I20260810 22:43:11.534698 9562 leader_election.cc:288] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [CANDIDATE]: Term 1 election: Requested vote from peers c83ad6b836f1468aadbd77af5f3c264b (127.19.202.65:34907), 86aa678410124422bb0492f70022acae (127.19.202.66:33805)
I20260810 22:43:11.534672 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa2ee8d77344efe8cbde9a78958847b" candidate_uuid: "86aa678410124422bb0492f70022acae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83ad6b836f1468aadbd77af5f3c264b" is_pre_election: true
I20260810 22:43:11.534816 9170 raft_consensus.cc:2454] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86aa678410124422bb0492f70022acae in term 0.
I20260810 22:43:11.535009 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa2ee8d77344efe8cbde9a78958847b" candidate_uuid: "c7377191f43e4b7482fda868ecd734a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83ad6b836f1468aadbd77af5f3c264b"
I20260810 22:43:11.535035 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa2ee8d77344efe8cbde9a78958847b" candidate_uuid: "c7377191f43e4b7482fda868ecd734a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86aa678410124422bb0492f70022acae"
I20260810 22:43:11.535115 9169 raft_consensus.cc:3037] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:11.535145 9320 raft_consensus.cc:3037] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:11.535116 9248 leader_election.cc:302] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86aa678410124422bb0492f70022acae, c83ad6b836f1468aadbd77af5f3c264b; no voters: c7377191f43e4b7482fda868ecd734a2
I20260810 22:43:11.536305 9320 raft_consensus.cc:2454] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7377191f43e4b7482fda868ecd734a2 in term 1.
I20260810 22:43:11.536334 9169 raft_consensus.cc:2454] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7377191f43e4b7482fda868ecd734a2 in term 1.
I20260810 22:43:11.536448 9560 raft_consensus.cc:2741] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 22:43:11.536710 9398 leader_election.cc:302] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86aa678410124422bb0492f70022acae, c7377191f43e4b7482fda868ecd734a2; no voters:
I20260810 22:43:11.536908 9562 raft_consensus.cc:2781] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:11.537156 9562 raft_consensus.cc:686] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 1 LEADER]: Becoming Leader. State: Replica: c7377191f43e4b7482fda868ecd734a2, State: Running, Role: LEADER
I20260810 22:43:11.537480 9562 consensus_queue.cc:237] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } }
I20260810 22:43:11.540512 8998 catalog_manager.cc:5495] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7377191f43e4b7482fda868ecd734a2 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "c7377191f43e4b7482fda868ecd734a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7377191f43e4b7482fda868ecd734a2" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 35555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 } health_report { overall_health: UNKNOWN } } }
I20260810 22:43:11.590133 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.586574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42118 (local address 127.19.202.67:35555)
0810 22:43:11.586723 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.586728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.586744 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:43:11.586844 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.586847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.586899 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.587016 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:11.587264 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.587751 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.588613 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.588870 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.588987 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.589291 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:11.589708 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:11.589834 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.589891 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.589955 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 22:43:11.596141 9169 raft_consensus.cc:1261] T 9aa2ee8d77344efe8cbde9a78958847b P c83ad6b836f1468aadbd77af5f3c264b [term 1 FOLLOWER]: Refusing update from remote peer c7377191f43e4b7482fda868ecd734a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:11.596143 9320 raft_consensus.cc:1261] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 1 FOLLOWER]: Refusing update from remote peer c7377191f43e4b7482fda868ecd734a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:11.596912 9569 consensus_queue.cc:1035] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86aa678410124422bb0492f70022acae" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 33805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:11.597121 9562 consensus_queue.cc:1035] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c83ad6b836f1468aadbd77af5f3c264b" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:11.605401 9575 mvcc.cc:205] Tried to move back new op lower bound from 7317101738370494464 to 7317101738139770880. Current Snapshot: MvccSnapshot[applied={T|T < 7317101738370494464}]
I20260810 22:43:11.605521 9576 mvcc.cc:205] Tried to move back new op lower bound from 7317101738370494464 to 7317101738139770880. Current Snapshot: MvccSnapshot[applied={T|T < 7317101738370494464}]
I20260810 22:43:11.612314 8997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:11.612473 8997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:11.614862 8997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8576eadb2f334d4ab73989284b28ec2d: Sending DeleteTablet for 3 replicas of tablet 9aa2ee8d77344efe8cbde9a78958847b
I20260810 22:43:11.615564 9456 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa2ee8d77344efe8cbde9a78958847b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:11 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:42104
I20260810 22:43:11.615779 9146 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa2ee8d77344efe8cbde9a78958847b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:11 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:46336
I20260810 22:43:11.615804 9298 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa2ee8d77344efe8cbde9a78958847b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:11 UTC) from {username='kudu', principal='kudu/127.19.202.126@KRBTEST.COM'} at 127.0.0.1:46390
I20260810 22:43:11.616214 9583 tablet_replica.cc:330] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: stopping tablet replica
I20260810 22:43:11.616253 9586 tablet_replica.cc:330] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: stopping tablet replica
I20260810 22:43:11.616513 9586 raft_consensus.cc:2229] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:11.616511 9583 raft_consensus.cc:2229] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:43:11.616882 9586 raft_consensus.cc:2258] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:11.616955 9583 raft_consensus.cc:2258] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:11.616976 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 9074
I20260810 22:43:11.619518 9583 ts_tablet_manager.cc:1905] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:11.619879 9586 ts_tablet_manager.cc:1905] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:11.622448 9583 ts_tablet_manager.cc:1918] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:11.622545 9583 log.cc:1196] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/9aa2ee8d77344efe8cbde9a78958847b
I20260810 22:43:11.622880 9583 ts_tablet_manager.cc:1939] T 9aa2ee8d77344efe8cbde9a78958847b P c7377191f43e4b7482fda868ecd734a2: Deleting consensus metadata
I20260810 22:43:11.623143 9586 ts_tablet_manager.cc:1918] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:11.623225 9586 log.cc:1196] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/9aa2ee8d77344efe8cbde9a78958847b
I20260810 22:43:11.623561 9586 ts_tablet_manager.cc:1939] T 9aa2ee8d77344efe8cbde9a78958847b P 86aa678410124422bb0492f70022acae: Deleting consensus metadata
I20260810 22:43:11.624166 8983 catalog_manager.cc:4841] TS c7377191f43e4b7482fda868ecd734a2 (127.19.202.67:35555): tablet 9aa2ee8d77344efe8cbde9a78958847b (table test-table [id=dbf57b149a78437dbf8b6f07992cb58c]) successfully deleted
I20260810 22:43:11.624687 8984 catalog_manager.cc:4841] TS 86aa678410124422bb0492f70022acae (127.19.202.66:33805): tablet 9aa2ee8d77344efe8cbde9a78958847b (table test-table [id=dbf57b149a78437dbf8b6f07992cb58c]) successfully deleted
W20260810 22:43:11.631527 8984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:34907 (error 108)
I20260810 22:43:11.632428 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 9226
I20260810 22:43:11.632587 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.632284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:34907 (local address 127.0.0.1:46352)
0810 22:43:11.632426 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:11.632499 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:34907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 22:43:11.634495 8984 catalog_manager.cc:4584] TS c83ad6b836f1468aadbd77af5f3c264b (127.19.202.65:34907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9aa2ee8d77344efe8cbde9a78958847b: Network error: Client connection negotiation failed: client connection to 127.19.202.65:34907: connect: Connection refused (error 111)
I20260810 22:43:11.641353 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 9378
I20260810 22:43:11.648595 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 8963
2026-08-10T22:43:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2612 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T22:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:11Z Disabled control of system clock
I20260810 22:43:11.683738 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39281
--webserver_interface=127.19.202.126
--webserver_port=0
--builtin_ntp_servers=127.19.202.84:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:39281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 22:43:11.797663 9593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:11.797946 9593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:11.797986 9593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:11.802599 9593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:11.802666 9593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:11.802685 9593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:11.802722 9593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:11.802762 9593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:11.807363 9593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:39281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:39281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:11.808522 9593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:11.810062 9593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:11.818595 9599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:11.818617 9600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:11.818617 9602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:11.818862 9593 server_base.cc:1034] running on GCE node
I20260810 22:43:11.819523 9593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:11.820629 9593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:11.821808 9593 hybrid_clock.cc:648] HybridClock initialized: now 1786401791821780 us; error 46 us; skew 500 ppm
I20260810 22:43:11.823607 9593 webserver.cc:460] Webserver started at http://127.19.202.126:33683/ using document root <none> and password file <none>
I20260810 22:43:11.824187 9593 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:11.824277 9593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:11.824506 9593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:11.826664 9593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "57665629f7e64737926d08da9cdc23e6"
format_stamp: "Formatted at 2026-08-10 22:43:11 on dist-test-slave-1x4g"
server_key: "ca97fbe77610b156d204eafb683b8157"
server_key_iv: "b77c94fd3878398a3d08c12c3a8001a3"
server_key_version: "encryptionkey@0"
I20260810 22:43:11.827311 9593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "57665629f7e64737926d08da9cdc23e6"
format_stamp: "Formatted at 2026-08-10 22:43:11 on dist-test-slave-1x4g"
server_key: "ca97fbe77610b156d204eafb683b8157"
server_key_iv: "b77c94fd3878398a3d08c12c3a8001a3"
server_key_version: "encryptionkey@0"
I20260810 22:43:11.831041 9593 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:43:11.833397 9610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:11.834553 9593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:11.834714 9593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "57665629f7e64737926d08da9cdc23e6"
format_stamp: "Formatted at 2026-08-10 22:43:11 on dist-test-slave-1x4g"
server_key: "ca97fbe77610b156d204eafb683b8157"
server_key_iv: "b77c94fd3878398a3d08c12c3a8001a3"
server_key_version: "encryptionkey@0"
I20260810 22:43:11.834812 9593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:11.850817 9593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:11.851467 9593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:11.851666 9593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:11.858852 9593 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:39281
I20260810 22:43:11.858858 9667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:39281 every 8 connection(s)
I20260810 22:43:11.859787 9593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:11.862921 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:11.868232 9668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6: Bootstrap starting.
I20260810 22:43:11.869958 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 9593
I20260810 22:43:11.870090 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:11.870398 20265 external_mini_cluster.cc:1428] Setting key e0bdd1cd5c3a9b7cf82ec0d14211ab7d
I20260810 22:43:11.870776 9668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:11.871714 9668 log.cc:824] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:11.873759 9668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6: No bootstrap required, opened a new log
I20260810 22:43:11.877509 9668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57665629f7e64737926d08da9cdc23e6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39281 } }
I20260810 22:43:11.877758 9668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:11.877837 9668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57665629f7e64737926d08da9cdc23e6, State: Initialized, Role: FOLLOWER
I20260810 22:43:11.878259 9668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57665629f7e64737926d08da9cdc23e6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39281 } }
I20260810 22:43:11.878412 9668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:11.878513 9668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:11.878635 9668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:11.879482 9668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57665629f7e64737926d08da9cdc23e6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39281 } }
I20260810 22:43:11.879899 9668 leader_election.cc:302] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57665629f7e64737926d08da9cdc23e6; no voters:
I20260810 22:43:11.880264 9668 leader_election.cc:288] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:11.880306 9673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:11.880511 9673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [term 1 LEADER]: Becoming Leader. State: Replica: 57665629f7e64737926d08da9cdc23e6, State: Running, Role: LEADER
I20260810 22:43:11.880816 9673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57665629f7e64737926d08da9cdc23e6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39281 } }
I20260810 22:43:11.881582 9668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:11.881986 9675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57665629f7e64737926d08da9cdc23e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57665629f7e64737926d08da9cdc23e6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39281 } } }
I20260810 22:43:11.882139 9675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:11.882359 9674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57665629f7e64737926d08da9cdc23e6. Latest consensus state: current_term: 1 leader_uuid: "57665629f7e64737926d08da9cdc23e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57665629f7e64737926d08da9cdc23e6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 39281 } } }
I20260810 22:43:11.882473 9674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:11.882784 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:11.871516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.19.202.126:39281)
0810 22:43:11.872240 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:11.872266 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:11.872312 (+ 46us) server_negotiation.cc:407] Connection header received
0810 22:43:11.873075 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:11.873093 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:11.873441 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:11.873759 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:11.874181 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.875151 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.876011 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:11.876509 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:11.876930 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:11.880183 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:11.880188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:11.880201 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:11.880708 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:11.880771 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:11.880823 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:11.881367 (+ 544us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:11.881427 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 22:43:11.881690 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":132,"threads_started":1}
I20260810 22:43:11.882853 9678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:11.885955 9678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:11.891239 9678 catalog_manager.cc:1347] Generated new cluster ID: 5cbab2f92c97499c897f1c2d6d9ca5eb
I20260810 22:43:11.891309 9678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:11.914314 9678 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:11.915539 9678 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:11.927690 9678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6: Generated new TSK 0
I20260810 22:43:11.928543 9678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:11.934989 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.65:0
--local_ip_for_outbound_sockets=127.19.202.65
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39281
--builtin_ntp_servers=127.19.202.84:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 22:43:12.069478 9696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:12.069815 9696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:12.069890 9696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:12.074296 9696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:12.074368 9696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:12.074456 9696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.65
I20260810 22:43:12.079466 9696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.202.65
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.65
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:12.080636 9696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:12.082185 9696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:12.091704 9701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:12.091691 9704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:12.091811 9696 server_base.cc:1034] running on GCE node
W20260810 22:43:12.091713 9702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:12.092408 9696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:12.093034 9696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:12.094194 9696 hybrid_clock.cc:648] HybridClock initialized: now 1786401792094180 us; error 26 us; skew 500 ppm
I20260810 22:43:12.095875 9696 webserver.cc:460] Webserver started at http://127.19.202.65:39175/ using document root <none> and password file <none>
I20260810 22:43:12.096390 9696 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:12.096447 9696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:12.096668 9696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:12.098757 9696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data/instance:
uuid: "b1536acf881e4020a3999e94349f05f3"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "ea4dd528237c49830fb64cdfb88f4b7c"
server_key_iv: "943d16f2e9dc6ec07daad14633a53c5a"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.099272 9696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance:
uuid: "b1536acf881e4020a3999e94349f05f3"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "ea4dd528237c49830fb64cdfb88f4b7c"
server_key_iv: "943d16f2e9dc6ec07daad14633a53c5a"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.102850 9696 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:43:12.105372 9713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.106560 9696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:12.106679 9696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/wal
uuid: "b1536acf881e4020a3999e94349f05f3"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "ea4dd528237c49830fb64cdfb88f4b7c"
server_key_iv: "943d16f2e9dc6ec07daad14633a53c5a"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.106778 9696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:12.116370 9696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:12.117034 9696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:12.117214 9696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:12.117776 9696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:12.118692 9696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:12.118737 9696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.118800 9696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:12.118830 9696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.127478 9696 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.65:37935
I20260810 22:43:12.127542 9846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.65:37935 every 8 connection(s)
I20260810 22:43:12.128396 9696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/data/info.pb
I20260810 22:43:12.134407 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 9696
I20260810 22:43:12.134541 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-0/wal/instance
I20260810 22:43:12.134876 20265 external_mini_cluster.cc:1428] Setting key c067ff02095663a9259c66f592a56156
I20260810 22:43:12.137204 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.66:0
--local_ip_for_outbound_sockets=127.19.202.66
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39281
--builtin_ntp_servers=127.19.202.84:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 22:43:12.138087 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.130330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.65:54289 (local address 127.19.202.126:39281)
0810 22:43:12.130467 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.130471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.131516 (+ 1045us) server_negotiation.cc:407] Connection header received
0810 22:43:12.132925 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.132927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.132972 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.133042 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:12.133939 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.134476 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.135895 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.136147 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.136259 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.137549 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:12.137553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:12.137555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:12.137567 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:12.137590 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:12.137600 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:12.137669 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.137892 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.137956 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:43:12.139305 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.130709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39281 (local address 127.19.202.65:54289)
0810 22:43:12.131371 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.131411 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.132696 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.133200 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.133206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.133282 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:12.133791 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.133805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.134641 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.134644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.135766 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.135777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.136302 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.136306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.136522 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.137234 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:12.137237 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:12.137294 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:12.137386 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:12.137710 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:12.137722 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:12.137731 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.137875 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.138210 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260810 22:43:12.140641 9847 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39281
I20260810 22:43:12.140923 9847 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:12.141503 9847 heartbeater.cc:511] Master 127.19.202.126:39281 requested a full tablet report, sending...
I20260810 22:43:12.143020 9628 ts_manager.cc:194] Registered new tserver with Master: b1536acf881e4020a3999e94349f05f3 (127.19.202.65:37935)
I20260810 22:43:12.144222 9628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.65:54289
W20260810 22:43:12.252507 9851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:12.252840 9851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:12.252934 9851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:12.256860 9851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:12.256948 9851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:12.257050 9851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.66
I20260810 22:43:12.261782 9851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.202.66
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.66
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:12.262874 9851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:12.264281 9851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:12.273677 9860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:12.273696 9859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:12.273933 9851 server_base.cc:1034] running on GCE node
W20260810 22:43:12.273679 9863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:12.274470 9851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:12.275104 9851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:12.276279 9851 hybrid_clock.cc:648] HybridClock initialized: now 1786401792276264 us; error 26 us; skew 500 ppm
I20260810 22:43:12.278265 9851 webserver.cc:460] Webserver started at http://127.19.202.66:38845/ using document root <none> and password file <none>
I20260810 22:43:12.278851 9851 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:12.278939 9851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:12.279145 9851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:12.281342 9851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data/instance:
uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "93cc6bae2eb385a78436a1fb812b7b27"
server_key_iv: "0fd15eea9c57d5bb5885cfefcf9a92c3"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.281944 9851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance:
uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "93cc6bae2eb385a78436a1fb812b7b27"
server_key_iv: "0fd15eea9c57d5bb5885cfefcf9a92c3"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.285724 9851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:12.288102 9870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.289332 9851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:12.289458 9851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal
uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "93cc6bae2eb385a78436a1fb812b7b27"
server_key_iv: "0fd15eea9c57d5bb5885cfefcf9a92c3"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.289569 9851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:12.304159 9851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:12.304939 9851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:12.305119 9851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:12.305701 9851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:12.306600 9851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:12.306664 9851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.306727 9851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:12.306758 9851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.315675 9851 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.66:37525
I20260810 22:43:12.315707 10002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.66:37525 every 8 connection(s)
I20260810 22:43:12.316541 9851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/data/info.pb
I20260810 22:43:12.325260 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.318422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.66:44637 (local address 127.19.202.126:39281)
0810 22:43:12.318546 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.318550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.319515 (+ 965us) server_negotiation.cc:407] Connection header received
0810 22:43:12.320929 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.320932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.320980 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.321055 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:12.321851 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.322353 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.323463 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.323674 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.323748 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.324863 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:12.324866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:12.324868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:12.324878 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:12.324896 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:12.324903 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:12.324958 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.325097 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.325146 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:43:12.326079 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.318747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39281 (local address 127.19.202.66:44637)
0810 22:43:12.319367 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.319408 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.320688 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.321183 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.321189 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.321268 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:12.321738 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.321750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.322455 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.322458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.323360 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.323370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.323795 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.323798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.323950 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.324588 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:12.324590 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:12.324630 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:12.324692 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:12.324973 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:12.324981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:12.324986 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.325083 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.325299 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":117,"threads_started":1}
I20260810 22:43:12.326718 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 9851
I20260810 22:43:12.326897 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal/instance
I20260810 22:43:12.327201 20265 external_mini_cluster.cc:1428] Setting key b9e641840499af8dae1c8bd1ab01510d
I20260810 22:43:12.327282 10003 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39281
I20260810 22:43:12.327512 10003 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:12.328053 10003 heartbeater.cc:511] Master 127.19.202.126:39281 requested a full tablet report, sending...
I20260810 22:43:12.329180 9628 ts_manager.cc:194] Registered new tserver with Master: 3960eaf53bf84bb38fe0b1bdee1b5e8f (127.19.202.66:37525)
I20260810 22:43:12.329319 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.202.67:0
--local_ip_for_outbound_sockets=127.19.202.67
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39281
--builtin_ntp_servers=127.19.202.84:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 22:43:12.329972 9628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.66:44637
W20260810 22:43:12.444099 10007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:12.444348 10007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:12.444384 10007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:12.448308 10007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:12.448364 10007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:12.448416 10007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.202.67
I20260810 22:43:12.453015 10007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.202.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.202.67
--webserver_port=0
--tserver_master_addrs=127.19.202.126:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.202.67
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:12.453997 10007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:12.455320 10007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:12.464879 10013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:12.464929 10014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:12.464922 10016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:12.465147 10007 server_base.cc:1034] running on GCE node
I20260810 22:43:12.465588 10007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:12.466251 10007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:12.467444 10007 hybrid_clock.cc:648] HybridClock initialized: now 1786401792467432 us; error 31 us; skew 500 ppm
I20260810 22:43:12.469362 10007 webserver.cc:460] Webserver started at http://127.19.202.67:46541/ using document root <none> and password file <none>
I20260810 22:43:12.469956 10007 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:12.470050 10007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:12.470310 10007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:12.472424 10007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data/instance:
uuid: "d6af51f61c5a42d5a3d2899e6228a2d3"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "72bce510f09a127f7e563f77856c446c"
server_key_iv: "ca53f37ac59e546bc614c8027dc7e7ee"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.473047 10007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance:
uuid: "d6af51f61c5a42d5a3d2899e6228a2d3"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "72bce510f09a127f7e563f77856c446c"
server_key_iv: "ca53f37ac59e546bc614c8027dc7e7ee"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.476945 10007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 22:43:12.479480 10026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.480707 10007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:12.480887 10007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal
uuid: "d6af51f61c5a42d5a3d2899e6228a2d3"
format_stamp: "Formatted at 2026-08-10 22:43:12 on dist-test-slave-1x4g"
server_key: "72bce510f09a127f7e563f77856c446c"
server_key_iv: "ca53f37ac59e546bc614c8027dc7e7ee"
server_key_version: "encryptionkey@0"
I20260810 22:43:12.481002 10007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:12.500545 10007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:12.501248 10007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:12.501446 10007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:12.502020 10007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:43:12.502962 10007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:43:12.503029 10007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.503096 10007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:43:12.503134 10007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:12.511798 10007 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.67:42453
I20260810 22:43:12.511897 10157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.67:42453 every 8 connection(s)
I20260810 22:43:12.512712 10007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/data/info.pb
I20260810 22:43:12.516499 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 10007
I20260810 22:43:12.516649 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal/instance
I20260810 22:43:12.516992 20265 external_mini_cluster.cc:1428] Setting key 5896cf3adab03855547c155daf466e46
I20260810 22:43:12.521780 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.514528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:37743 (local address 127.19.202.126:39281)
0810 22:43:12.514674 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.514678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.515632 (+ 954us) server_negotiation.cc:407] Connection header received
0810 22:43:12.517251 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.517255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.517315 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.517424 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:12.518301 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.518938 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.520096 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.520313 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.520390 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.521380 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:12.521383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:12.521385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:12.521396 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:12.521422 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:12.521431 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:12.521488 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.521628 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.521674 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 22:43:12.522598 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.514869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:39281 (local address 127.19.202.67:37743)
0810 22:43:12.515490 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.515529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.517018 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.517599 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.517606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.517688 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:12.518166 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.518179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.519076 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.519079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.519989 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.519999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.520437 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.520440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.520577 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.521149 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:43:12.521150 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:12.521189 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:43:12.521251 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:12.521499 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:12.521507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:12.521511 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.521615 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.521839 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
I20260810 22:43:12.523648 10158 heartbeater.cc:347] Connected to a master server at 127.19.202.126:39281
I20260810 22:43:12.523871 10158 heartbeater.cc:464] Registering TS with master...
I20260810 22:43:12.524358 10158 heartbeater.cc:511] Master 127.19.202.126:39281 requested a full tablet report, sending...
I20260810 22:43:12.525415 9628 ts_manager.cc:194] Registered new tserver with Master: d6af51f61c5a42d5a3d2899e6228a2d3 (127.19.202.67:42453)
I20260810 22:43:12.525947 9628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.202.67:37743
I20260810 22:43:12.530551 20265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:43:12.535701 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.532398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.19.202.126:39281)
0810 22:43:12.532513 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.532516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.532685 (+ 169us) server_negotiation.cc:407] Connection header received
0810 22:43:12.532818 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.532821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.532864 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.532939 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:12.533148 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.533604 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.534386 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.534605 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.534688 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.535359 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:12.535362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:12.535364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:12.535374 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:12.535391 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:12.535398 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:12.535456 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.535557 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.535601 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:43:12.538269 9628 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:43:12.541294 9628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:43:12.555166 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.549387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:37525 (local address 127.0.0.1:58944)
0810 22:43:12.549885 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.549935 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.550066 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.551438 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.551441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.551461 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:12.551642 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.551651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.552585 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.552590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.553795 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.553805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.554634 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.554637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.554744 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.554781 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.554888 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.555014 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":83,"thread_start_us":87,"threads_started":1}
I20260810 22:43:12.555220 10173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.549540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.19.202.66:37525)
0810 22:43:12.550910 (+ 1370us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.550922 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.550942 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:43:12.551008 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.551013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.551160 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.551288 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:12.551766 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.552479 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.553918 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.554519 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.554626 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.554810 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.554948 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.555055 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1260,"thread_start_us":137,"threads_started":1}
I20260810 22:43:12.555734 10175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.550237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.67:42453 (local address 127.0.0.1:40282)
0810 22:43:12.550591 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.550604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.550689 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.551268 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.551276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.551304 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:12.551488 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.551495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.553041 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.553045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.554285 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.554296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.555333 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.555336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.555459 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.555486 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.555547 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.555596 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":77,"thread_start_us":105,"threads_started":1}
I20260810 22:43:12.556106 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.549582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:37935 (local address 127.0.0.1:58284)
0810 22:43:12.550042 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.550056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.550162 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.551362 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.551364 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.551378 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:12.551613 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.551621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.553041 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.553045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.554345 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.554354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.555728 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.555731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.555825 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.555860 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.555923 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.555992 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":206,"thread_start_us":91,"threads_started":1}
I20260810 22:43:12.556495 10176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.550465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.19.202.67:42453)
0810 22:43:12.550769 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.550781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.550807 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:43:12.550886 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.550891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.551093 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.551260 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:12.551628 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.552896 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.554419 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.555190 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.555989 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.556167 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.556220 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.556306 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":117,"threads_started":1}
I20260810 22:43:12.556833 10174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.550194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.19.202.65:37935)
0810 22:43:12.550953 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.550963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.550983 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:43:12.551047 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.551051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.551214 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.551491 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:12.551730 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.552889 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.554465 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.555213 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.555727 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.556281 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.556345 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.556451 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":119,"threads_started":1}
I20260810 22:43:12.558620 9922 tablet_service.cc:1467] Processing CreateTablet for tablet 0014a63eac834d20b1793347cdebadeb (DEFAULT_TABLE table=test-table [id=b39755e2ce1e48818cd22d33a392380d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:12.559043 10081 tablet_service.cc:1467] Processing CreateTablet for tablet 0014a63eac834d20b1793347cdebadeb (DEFAULT_TABLE table=test-table [id=b39755e2ce1e48818cd22d33a392380d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:12.559633 9922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0014a63eac834d20b1793347cdebadeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:12.559597 9772 tablet_service.cc:1467] Processing CreateTablet for tablet 0014a63eac834d20b1793347cdebadeb (DEFAULT_TABLE table=test-table [id=b39755e2ce1e48818cd22d33a392380d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:43:12.560029 10081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0014a63eac834d20b1793347cdebadeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:12.560540 9772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0014a63eac834d20b1793347cdebadeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:12.565966 10177 tablet_bootstrap.cc:492] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Bootstrap starting.
I20260810 22:43:12.567945 10181 tablet_bootstrap.cc:492] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: Bootstrap starting.
I20260810 22:43:12.568132 10177 tablet_bootstrap.cc:654] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:12.568877 10177 log.cc:824] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:12.569100 10178 tablet_bootstrap.cc:492] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Bootstrap starting.
I20260810 22:43:12.569944 10181 tablet_bootstrap.cc:654] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:12.570768 10177 tablet_bootstrap.cc:492] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: No bootstrap required, opened a new log
I20260810 22:43:12.570756 10181 log.cc:824] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:12.570957 10177 ts_tablet_manager.cc:1397] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:43:12.571156 10178 tablet_bootstrap.cc:654] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:12.571842 10178 log.cc:824] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:12.572674 10181 tablet_bootstrap.cc:492] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: No bootstrap required, opened a new log
I20260810 22:43:12.572880 10181 ts_tablet_manager.cc:1397] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 22:43:12.573427 10178 tablet_bootstrap.cc:492] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: No bootstrap required, opened a new log
I20260810 22:43:12.573635 10178 ts_tablet_manager.cc:1397] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 22:43:12.575313 10177 raft_consensus.cc:348] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.575582 10177 raft_consensus.cc:374] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:12.575639 10177 raft_consensus.cc:729] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3960eaf53bf84bb38fe0b1bdee1b5e8f, State: Initialized, Role: FOLLOWER
I20260810 22:43:12.576102 10177 consensus_queue.cc:260] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.576957 10178 raft_consensus.cc:348] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.577154 10177 ts_tablet_manager.cc:1428] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 22:43:12.577188 10003 heartbeater.cc:503] Master 127.19.202.126:39281 was elected leader, sending a full tablet report...
I20260810 22:43:12.577203 10178 raft_consensus.cc:374] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:12.577076 10181 raft_consensus.cc:348] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.577251 10178 raft_consensus.cc:729] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6af51f61c5a42d5a3d2899e6228a2d3, State: Initialized, Role: FOLLOWER
I20260810 22:43:12.577306 10181 raft_consensus.cc:374] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:12.577361 10181 raft_consensus.cc:729] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1536acf881e4020a3999e94349f05f3, State: Initialized, Role: FOLLOWER
I20260810 22:43:12.577688 10178 consensus_queue.cc:260] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.577996 10181 consensus_queue.cc:260] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.578611 10178 ts_tablet_manager.cc:1428] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 22:43:12.578716 10158 heartbeater.cc:503] Master 127.19.202.126:39281 was elected leader, sending a full tablet report...
I20260810 22:43:12.579082 10181 ts_tablet_manager.cc:1428] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 22:43:12.579417 9847 heartbeater.cc:503] Master 127.19.202.126:39281 was elected leader, sending a full tablet report...
W20260810 22:43:12.629647 9849 tablet.cc:2369] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:43:12.763720 10159 tablet.cc:2369] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:43:12.786415 10188 raft_consensus.cc:482] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:43:12.786638 10188 raft_consensus.cc:504] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.787784 10188 leader_election.cc:288] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3960eaf53bf84bb38fe0b1bdee1b5e8f (127.19.202.66:37525), b1536acf881e4020a3999e94349f05f3 (127.19.202.65:37935)
I20260810 22:43:12.792721 10174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.788152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:47079 (local address 127.19.202.65:37935)
0810 22:43:12.788287 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.788292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.788309 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:43:12.788439 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.788442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.788497 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.788578 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:12.788975 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.789754 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.791394 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.791981 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.792111 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.792160 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.792501 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.792579 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:43:12.792721 10173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.788162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.202.67:34863 (local address 127.19.202.66:37525)
0810 22:43:12.788287 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.788291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.788698 (+ 407us) server_negotiation.cc:407] Connection header received
0810 22:43:12.788888 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.788891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.788958 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.789040 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:43:12.789335 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.789980 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.791388 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.792006 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.792111 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.792160 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.792512 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.792582 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 22:43:12.792775 10190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.788021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.66:37525 (local address 127.19.202.67:34863)
0810 22:43:12.788614 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.788628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.788734 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.789063 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.789065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.789083 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:12.789204 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.789209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.790088 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.790091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.791248 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.791257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.792121 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.792125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.792345 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.792392 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.792472 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.792585 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":302,"thread_start_us":161,"threads_started":1}
I20260810 22:43:12.792766 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.788021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:37935 (local address 127.19.202.67:47079)
0810 22:43:12.788166 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.788185 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:12.788289 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:12.788617 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:12.788622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:12.788645 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:43:12.788843 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.788854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.789888 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.789891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.791219 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:12.791232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.792121 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.792125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:12.792341 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.792388 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 22:43:12.792476 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 22:43:12.792586 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":65}
I20260810 22:43:12.793613 9946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0014a63eac834d20b1793347cdebadeb" candidate_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" is_pre_election: true
I20260810 22:43:12.794000 9946 raft_consensus.cc:2454] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6af51f61c5a42d5a3d2899e6228a2d3 in term 0.
I20260810 22:43:12.794067 9795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0014a63eac834d20b1793347cdebadeb" candidate_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1536acf881e4020a3999e94349f05f3" is_pre_election: true
I20260810 22:43:12.794335 9795 raft_consensus.cc:2454] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6af51f61c5a42d5a3d2899e6228a2d3 in term 0.
I20260810 22:43:12.794551 10027 leader_election.cc:302] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3960eaf53bf84bb38fe0b1bdee1b5e8f, d6af51f61c5a42d5a3d2899e6228a2d3; no voters:
I20260810 22:43:12.794865 10188 raft_consensus.cc:2781] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:43:12.794979 10188 raft_consensus.cc:482] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:43:12.795037 10188 raft_consensus.cc:3037] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:12.796712 10188 raft_consensus.cc:504] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.797295 10188 leader_election.cc:288] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 3960eaf53bf84bb38fe0b1bdee1b5e8f (127.19.202.66:37525), b1536acf881e4020a3999e94349f05f3 (127.19.202.65:37935)
I20260810 22:43:12.797645 9795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0014a63eac834d20b1793347cdebadeb" candidate_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1536acf881e4020a3999e94349f05f3"
I20260810 22:43:12.797691 9946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0014a63eac834d20b1793347cdebadeb" candidate_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f"
I20260810 22:43:12.797806 9795 raft_consensus.cc:3037] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:12.797885 9946 raft_consensus.cc:3037] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:12.798972 9795 raft_consensus.cc:2454] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6af51f61c5a42d5a3d2899e6228a2d3 in term 1.
I20260810 22:43:12.799270 10031 leader_election.cc:302] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1536acf881e4020a3999e94349f05f3, d6af51f61c5a42d5a3d2899e6228a2d3; no voters:
I20260810 22:43:12.799335 9946 raft_consensus.cc:2454] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6af51f61c5a42d5a3d2899e6228a2d3 in term 1.
I20260810 22:43:12.799461 10188 raft_consensus.cc:2781] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:12.799723 10188 raft_consensus.cc:686] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 1 LEADER]: Becoming Leader. State: Replica: d6af51f61c5a42d5a3d2899e6228a2d3, State: Running, Role: LEADER
I20260810 22:43:12.800046 10188 consensus_queue.cc:237] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } }
I20260810 22:43:12.803184 9627 catalog_manager.cc:5495] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6af51f61c5a42d5a3d2899e6228a2d3 (127.19.202.67). New cstate: current_term: 1 leader_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6af51f61c5a42d5a3d2899e6228a2d3" member_type: VOTER last_known_addr { host: "127.19.202.67" port: 42453 } health_report { overall_health: HEALTHY } } }
W20260810 22:43:12.817759 10005 tablet.cc:2369] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:43:12.847113 10176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.843509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40298 (local address 127.19.202.67:42453)
0810 22:43:12.843634 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:12.843638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:12.843655 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:43:12.843772 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:12.843774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:12.843831 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:12.843939 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:12.844185 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.844712 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.845576 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:12.845823 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:12.845934 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:12.846259 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:12.846695 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:12.846828 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:12.846892 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 22:43:12.846947 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:43:12.853452 9795 raft_consensus.cc:1261] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3 [term 1 FOLLOWER]: Refusing update from remote peer d6af51f61c5a42d5a3d2899e6228a2d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:12.853597 9946 raft_consensus.cc:1261] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 1 FOLLOWER]: Refusing update from remote peer d6af51f61c5a42d5a3d2899e6228a2d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:12.854192 10188 consensus_queue.cc:1035] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1536acf881e4020a3999e94349f05f3" member_type: VOTER last_known_addr { host: "127.19.202.65" port: 37935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:12.854656 10191 consensus_queue.cc:1035] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3960eaf53bf84bb38fe0b1bdee1b5e8f" member_type: VOTER last_known_addr { host: "127.19.202.66" port: 37525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:12.863713 10196 mvcc.cc:205] Tried to move back new op lower bound from 7317101743520333824 to 7317101743311728640. Current Snapshot: MvccSnapshot[applied={T|T < 7317101743520333824}]
I20260810 22:43:12.870884 9627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:12.871065 9627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:43:12.873875 9627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57665629f7e64737926d08da9cdc23e6: Sending DeleteTablet for 3 replicas of tablet 0014a63eac834d20b1793347cdebadeb
I20260810 22:43:12.874626 9922 tablet_service.cc:1514] Processing DeleteTablet for tablet 0014a63eac834d20b1793347cdebadeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:12 UTC) from {username='slave'} at 127.0.0.1:58944
I20260810 22:43:12.874666 9772 tablet_service.cc:1514] Processing DeleteTablet for tablet 0014a63eac834d20b1793347cdebadeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:12 UTC) from {username='slave'} at 127.0.0.1:58284
I20260810 22:43:12.875089 10203 tablet_replica.cc:330] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: stopping tablet replica
I20260810 22:43:12.875451 10203 raft_consensus.cc:2229] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:43:12.875581 10081 tablet_service.cc:1514] Processing DeleteTablet for tablet 0014a63eac834d20b1793347cdebadeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:43:12 UTC) from {username='slave'} at 127.0.0.1:40282
I20260810 22:43:12.875828 10203 raft_consensus.cc:2258] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:12.876155 10195 mvcc.cc:205] Tried to move back new op lower bound from 7317101743520333824 to 7317101743311728640. Current Snapshot: MvccSnapshot[applied={T|T < 7317101743520333824}]
I20260810 22:43:12.876279 10204 tablet_replica.cc:330] T 0014a63eac834d20b1793347cdebadeb P b1536acf881e4020a3999e94349f05f3: stopping tablet replica
I20260810 22:43:12.876536 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 9696
I20260810 22:43:12.881497 10205 tablet_replica.cc:330] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: stopping tablet replica
I20260810 22:43:12.881923 10205 raft_consensus.cc:2229] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:43:12.882328 10205 raft_consensus.cc:2258] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:43:12.887385 10203 ts_tablet_manager.cc:1905] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 22:43:12.889817 9614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.202.65:37935 (error 108)
I20260810 22:43:12.890389 10203 ts_tablet_manager.cc:1918] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:12.890497 10203 log.cc:1196] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-1/wal/wals/0014a63eac834d20b1793347cdebadeb
I20260810 22:43:12.890828 10203 ts_tablet_manager.cc:1939] T 0014a63eac834d20b1793347cdebadeb P 3960eaf53bf84bb38fe0b1bdee1b5e8f: Deleting consensus metadata
I20260810 22:43:12.891579 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:12.891155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.65:37935 (local address 127.0.0.1:58298)
0810 22:43:12.891392 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:12.891475 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.65:37935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260810 22:43:12.891774 9614 catalog_manager.cc:4584] TS b1536acf881e4020a3999e94349f05f3 (127.19.202.65:37935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0014a63eac834d20b1793347cdebadeb: Network error: Client connection negotiation failed: client connection to 127.19.202.65:37935: connect: Connection refused (error 111)
I20260810 22:43:12.892131 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 9851
I20260810 22:43:12.892109 9611 catalog_manager.cc:4841] TS 3960eaf53bf84bb38fe0b1bdee1b5e8f (127.19.202.66:37525): tablet 0014a63eac834d20b1793347cdebadeb (table test-table [id=b39755e2ce1e48818cd22d33a392380d]) successfully deleted
I20260810 22:43:12.895519 10205 ts_tablet_manager.cc:1905] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:43:12.898288 10205 ts_tablet_manager.cc:1918] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:43:12.898407 10205 log.cc:1196] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Deleting WAL directory at /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786401536977653-20265-0/minicluster-data/ts-2/wal/wals/0014a63eac834d20b1793347cdebadeb
I20260810 22:43:12.898806 10205 ts_tablet_manager.cc:1939] T 0014a63eac834d20b1793347cdebadeb P d6af51f61c5a42d5a3d2899e6228a2d3: Deleting consensus metadata
I20260810 22:43:12.900394 9613 catalog_manager.cc:4841] TS d6af51f61c5a42d5a3d2899e6228a2d3 (127.19.202.67:42453): tablet 0014a63eac834d20b1793347cdebadeb (table test-table [id=b39755e2ce1e48818cd22d33a392380d]) successfully deleted
I20260810 22:43:12.903437 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 10007
I20260810 22:43:12.915654 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 9593
2026-08-10T22:43:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1267 ms)
[----------] 6 tests from EncryptionPolicyTest (11164 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:12 dist-test-slave-1x4g krb5kdc[10210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:12 dist-test-slave-1x4g krb5kdc[10210](info): set up 2 sockets
Aug 10 22:43:12 dist-test-slave-1x4g krb5kdc[10210](info): commencing operation
krb5kdc: starting...
W20260810 22:43:14.003731 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:14 dist-test-slave-1x4g krb5kdc[10210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:14.127379 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:14.238642 10226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:14.238912 10226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:14.238987 10226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:14.243125 10226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:14.243182 10226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:14.243201 10226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:14.243216 10226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:14.243230 10226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:14.248348 10226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:14.249358 10226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:14.250422 10226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:43:14.259137 10226 server_base.cc:1034] running on GCE node
W20260810 22:43:14.259167 10231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:14.259370 10234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:14.259385 10232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:14.260058 10226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:14.261234 10226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:14.262419 10226 hybrid_clock.cc:648] HybridClock initialized: now 1786401794262394 us; error 37 us; skew 500 ppm
Aug 10 22:43:14 dist-test-slave-1x4g krb5kdc[10210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:14.265200 10226 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:14.266072 10226 webserver.cc:460] Webserver started at http://127.0.0.1:37121/ using document root <none> and password file <none>
I20260810 22:43:14.266640 10226 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:14.266726 10226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:14.266958 10226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:14.269035 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "0c7e2d861acb48a080b7f59da21ad7bd"
format_stamp: "Formatted at 2026-08-10 22:43:14 on dist-test-slave-1x4g"
server_key: "6c6cb370db090f487bd170a63aa1576d"
server_key_iv: "d10b10d40d324d158a6441fa785e6f17"
server_key_version: "encryptionkey@0"
I20260810 22:43:14.269587 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "0c7e2d861acb48a080b7f59da21ad7bd"
format_stamp: "Formatted at 2026-08-10 22:43:14 on dist-test-slave-1x4g"
server_key: "6c6cb370db090f487bd170a63aa1576d"
server_key_iv: "d10b10d40d324d158a6441fa785e6f17"
server_key_version: "encryptionkey@0"
I20260810 22:43:14.273241 10226 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:43:14.275686 10242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:14.276969 10226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:14.277097 10226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "0c7e2d861acb48a080b7f59da21ad7bd"
format_stamp: "Formatted at 2026-08-10 22:43:14 on dist-test-slave-1x4g"
server_key: "6c6cb370db090f487bd170a63aa1576d"
server_key_iv: "d10b10d40d324d158a6441fa785e6f17"
server_key_version: "encryptionkey@0"
I20260810 22:43:14.277218 10226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:14.299448 10226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:14.302593 10226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:14.302814 10226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:14.310004 10226 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36593
I20260810 22:43:14.310067 10303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36593 every 8 connection(s)
I20260810 22:43:14.311261 10226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:14.313934 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 10226
I20260810 22:43:14.314079 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:14.314393 20265 external_mini_cluster.cc:1428] Setting key 4646995af123256251fb5a8c108b7d47
I20260810 22:43:14.314486 10304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:14.321458 10304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd: Bootstrap starting.
I20260810 22:43:14.324278 10304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd: Neither blocks nor log segments found. Creating new log.
Aug 10 22:43:14 dist-test-slave-1x4g krb5kdc[10210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 22:43:14.325157 10304 log.cc:824] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:14.327178 10304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd: No bootstrap required, opened a new log
I20260810 22:43:14.330101 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:14.315733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44495 (local address 127.0.0.1:36593)
0810 22:43:14.316155 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:14.316174 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:14.317130 (+ 956us) server_negotiation.cc:407] Connection header received
0810 22:43:14.317909 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:14.317926 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:14.318243 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:14.318522 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:14.319421 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:14.320435 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:14.321662 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:14.322138 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:14.322257 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:14.325262 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:14.325271 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:14.325289 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:14.325325 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:14.327204 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:14.327736 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:14.327741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:14.327747 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:14.327859 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:14.328222 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:14.328225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:14.328227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:14.328612 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:14.328851 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:14.329178 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 22:43:14.329405 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260810 22:43:14.330834 10304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 22:43:14.331086 10304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:14.331151 10304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7e2d861acb48a080b7f59da21ad7bd, State: Initialized, Role: FOLLOWER
I20260810 22:43:14.331619 10304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 22:43:14.331768 10304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:14.331830 10304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:14.331944 10304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:14.333141 10304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 22:43:14.333279 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:14.333487 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c7e2d861acb48a080b7f59da21ad7bd; no voters:
I20260810 22:43:14.333804 10304 leader_election.cc:288] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:14.333875 10310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:14.334075 10310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [term 1 LEADER]: Becoming Leader. State: Replica: 0c7e2d861acb48a080b7f59da21ad7bd, State: Running, Role: LEADER
I20260810 22:43:14.334352 10310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 22:43:14.335071 10304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:14.335964 10312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } } }
I20260810 22:43:14.336119 10312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:14.336428 10311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c7e2d861acb48a080b7f59da21ad7bd. Latest consensus state: current_term: 1 leader_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e2d861acb48a080b7f59da21ad7bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } } }
I20260810 22:43:14.336520 10311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:14.336526 10325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:14.339701 10325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:14.345486 10325 catalog_manager.cc:1347] Generated new cluster ID: 634290c314854226a43906d57dd2dcc2
I20260810 22:43:14.345549 10325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:14.346480 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:14.335386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41865 (local address 127.0.0.1:36593)
0810 22:43:14.335499 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:14.335503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:14.337075 (+ 1572us) server_negotiation.cc:407] Connection header received
0810 22:43:14.337277 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:14.337280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:14.337329 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:14.337400 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:14.338197 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:14.338942 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:14.340112 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:14.340449 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:14.340557 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:14.342289 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:14.342293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:14.342296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:14.342319 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:14.344015 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:14.344668 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:14.344671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:14.344673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:14.344763 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:14.345420 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:14.345448 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:14.345449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:14.345675 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:14.345836 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:14.346202 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 22:43:14.346321 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:43:14.365850 10325 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:14.366775 10325 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:14.374465 10325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c7e2d861acb48a080b7f59da21ad7bd: Generated new TSK 0
I20260810 22:43:14.375316 10325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:14.388334 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 10226
W20260810 22:43:14.397702 10208 connection.cc:535] client connection to 127.0.0.1:36593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36593): Connection reset by peer (error 104)
2026-08-10T22:43:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:14 dist-test-slave-1x4g krb5kdc[10339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:14 dist-test-slave-1x4g krb5kdc[10339](info): set up 2 sockets
Aug 10 22:43:14 dist-test-slave-1x4g krb5kdc[10339](info): commencing operation
krb5kdc: starting...
W20260810 22:43:15.458889 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:15 dist-test-slave-1x4g krb5kdc[10339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:15.580150 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:15.688544 10355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:15.688836 10355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:15.688939 10355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:15.692754 10355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:15.692848 10355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:15.692898 10355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:15.692943 10355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:15.692981 10355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:15.698011 10355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:15.698949 10355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:15.699962 10355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:15.708171 10362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:15.708186 10361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:15.708191 10364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:15.708534 10355 server_base.cc:1034] running on GCE node
I20260810 22:43:15.709115 10355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:15.710176 10355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:15.711339 10355 hybrid_clock.cc:648] HybridClock initialized: now 1786401795711321 us; error 31 us; skew 500 ppm
Aug 10 22:43:15 dist-test-slave-1x4g krb5kdc[10339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401795, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:15.714128 10355 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:15.714936 10355 webserver.cc:460] Webserver started at http://127.0.0.1:45895/ using document root <none> and password file <none>
I20260810 22:43:15.715438 10355 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:15.715499 10355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:15.715660 10355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:15.717785 10355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "3756a6ab934b4ba0b26042173de00ff1"
format_stamp: "Formatted at 2026-08-10 22:43:15 on dist-test-slave-1x4g"
server_key: "010534e8fee9c20256f386d455d4c2a1"
server_key_iv: "b05e076a7f53b07876b7d3ce06b47d5f"
server_key_version: "encryptionkey@0"
I20260810 22:43:15.718307 10355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "3756a6ab934b4ba0b26042173de00ff1"
format_stamp: "Formatted at 2026-08-10 22:43:15 on dist-test-slave-1x4g"
server_key: "010534e8fee9c20256f386d455d4c2a1"
server_key_iv: "b05e076a7f53b07876b7d3ce06b47d5f"
server_key_version: "encryptionkey@0"
I20260810 22:43:15.722050 10355 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:15.724433 10374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:15.725806 10355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:43:15.725948 10355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "3756a6ab934b4ba0b26042173de00ff1"
format_stamp: "Formatted at 2026-08-10 22:43:15 on dist-test-slave-1x4g"
server_key: "010534e8fee9c20256f386d455d4c2a1"
server_key_iv: "b05e076a7f53b07876b7d3ce06b47d5f"
server_key_version: "encryptionkey@0"
I20260810 22:43:15.726065 10355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:15.744503 10355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:15.747318 10355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:15.747529 10355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:15.754634 10355 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39667
I20260810 22:43:15.754647 10436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39667 every 8 connection(s)
I20260810 22:43:15.755597 10355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:15.756645 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 10355
I20260810 22:43:15.756824 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:15.757174 20265 external_mini_cluster.cc:1428] Setting key 2b2f1ec2d4c3e8287cd9acfe7ffee88b
I20260810 22:43:15.758986 10437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:15.765615 10437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1: Bootstrap starting.
I20260810 22:43:15.766781 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:15.758492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35945 (local address 127.0.0.1:39667)
0810 22:43:15.758899 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:15.758926 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:15.758967 (+ 41us) server_negotiation.cc:407] Connection header received
0810 22:43:15.759762 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:15.759783 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:15.760188 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:15.760546 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:15.760961 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:15.762502 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:15.763426 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:15.764114 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:15.764267 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:15.765159 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:15.765164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:15.765176 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:15.765553 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:15.765604 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:15.765625 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:15.765763 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:15.765932 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 22:43:15.766080 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
I20260810 22:43:15.767858 10437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:15.768743 10437 log.cc:824] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:15.769757 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:15.770721 10437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1: No bootstrap required, opened a new log
I20260810 22:43:15.774314 10437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756a6ab934b4ba0b26042173de00ff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260810 22:43:15.774528 10437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:15.774611 10437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3756a6ab934b4ba0b26042173de00ff1, State: Initialized, Role: FOLLOWER
I20260810 22:43:15.775081 10437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756a6ab934b4ba0b26042173de00ff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260810 22:43:15.775275 10437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:15.775368 10437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:15.775475 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:15.771696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49691 (local address 127.0.0.1:39667)
0810 22:43:15.771797 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:15.771804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:15.771963 (+ 159us) server_negotiation.cc:407] Connection header received
0810 22:43:15.772097 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:15.772100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:15.772143 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:15.772219 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:15.772474 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:15.773013 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:15.773780 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:15.773998 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:15.774096 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:15.774916 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:15.774919 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:15.774922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:15.774934 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:15.774964 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:15.774974 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:15.775042 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:15.775215 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 22:43:15.775265 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 22:43:15.775513 10437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:15.776655 10437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756a6ab934b4ba0b26042173de00ff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260810 22:43:15.777024 10437 leader_election.cc:302] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3756a6ab934b4ba0b26042173de00ff1; no voters:
I20260810 22:43:15.777308 10437 leader_election.cc:288] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:15.777500 10451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:15.777856 10451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [term 1 LEADER]: Becoming Leader. State: Replica: 3756a6ab934b4ba0b26042173de00ff1, State: Running, Role: LEADER
I20260810 22:43:15.778285 10451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756a6ab934b4ba0b26042173de00ff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260810 22:43:15.778594 10437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:15.779847 10453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3756a6ab934b4ba0b26042173de00ff1. Latest consensus state: current_term: 1 leader_uuid: "3756a6ab934b4ba0b26042173de00ff1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756a6ab934b4ba0b26042173de00ff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } } }
I20260810 22:43:15.779994 10453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:15.780326 10452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3756a6ab934b4ba0b26042173de00ff1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756a6ab934b4ba0b26042173de00ff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } } }
I20260810 22:43:15.780421 10452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1 [sys.catalog]: This master's current role is: LEADER
W20260810 22:43:15.783388 10469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:15.783494 10469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:15.783588 10463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:15.784282 10463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:15.789063 10463 catalog_manager.cc:1347] Generated new cluster ID: b8d6a1e258844a0396f4415c9aff4233
I20260810 22:43:15.789125 10463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:15.800325 10463 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:15.801318 10463 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:15.805847 10463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3756a6ab934b4ba0b26042173de00ff1: Generated new TSK 0
I20260810 22:43:15.806427 10463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:15.810256 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 10355
W20260810 22:43:15.818364 10337 connection.cc:535] client connection to 127.0.0.1:39667 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39667): Connection reset by peer (error 104)
2026-08-10T22:43:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:15 dist-test-slave-1x4g krb5kdc[10475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:15 dist-test-slave-1x4g krb5kdc[10475](info): set up 2 sockets
Aug 10 22:43:15 dist-test-slave-1x4g krb5kdc[10475](info): commencing operation
krb5kdc: starting...
W20260810 22:43:16.879365 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:16 dist-test-slave-1x4g krb5kdc[10475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:17.003664 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:17.115234 10489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:17.115492 10489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:17.115530 10489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:17.119576 10489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:17.119642 10489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:17.119663 10489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:17.119679 10489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:17.119694 10489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:17.125409 10489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35227
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:17.126485 10489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:17.127439 10489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:17.135848 10495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:17.135862 10494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:17.136099 10489 server_base.cc:1034] running on GCE node
W20260810 22:43:17.136209 10498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:17.136762 10489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:17.137846 10489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:17.139078 10489 hybrid_clock.cc:648] HybridClock initialized: now 1786401797139035 us; error 33 us; skew 500 ppm
Aug 10 22:43:17 dist-test-slave-1x4g krb5kdc[10475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:17.147156 10489 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:17.148123 10489 webserver.cc:460] Webserver started at http://127.0.0.1:34859/ using document root <none> and password file <none>
I20260810 22:43:17.148696 10489 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:17.148813 10489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:17.149053 10489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:17.151063 10489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "404f86bb368f421589aababa27813d84"
format_stamp: "Formatted at 2026-08-10 22:43:17 on dist-test-slave-1x4g"
server_key: "34f6b148bda0e6e875e9dcc0ae86c5ef"
server_key_iv: "6fa7b580979ff4928075a235921bdb33"
server_key_version: "encryptionkey@0"
I20260810 22:43:17.151634 10489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "404f86bb368f421589aababa27813d84"
format_stamp: "Formatted at 2026-08-10 22:43:17 on dist-test-slave-1x4g"
server_key: "34f6b148bda0e6e875e9dcc0ae86c5ef"
server_key_iv: "6fa7b580979ff4928075a235921bdb33"
server_key_version: "encryptionkey@0"
I20260810 22:43:17.155576 10489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 22:43:17.158102 10505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:17.159344 10489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:17.159500 10489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "404f86bb368f421589aababa27813d84"
format_stamp: "Formatted at 2026-08-10 22:43:17 on dist-test-slave-1x4g"
server_key: "34f6b148bda0e6e875e9dcc0ae86c5ef"
server_key_iv: "6fa7b580979ff4928075a235921bdb33"
server_key_version: "encryptionkey@0"
I20260810 22:43:17.159617 10489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:17.164173 10489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:17.167649 10489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:17.167829 10489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:17.174887 10489 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35227
I20260810 22:43:17.174896 10567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35227 every 8 connection(s)
I20260810 22:43:17.175777 10489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:17.178599 10568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:17.180107 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 10489
I20260810 22:43:17.180250 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:17.180524 20265 external_mini_cluster.cc:1428] Setting key 1edc9b62978accc25fc3f6ea84acefc5
I20260810 22:43:17.184865 10568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84: Bootstrap starting.
I20260810 22:43:17.185627 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:17.181982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50501 (local address 127.0.0.1:35227)
0810 22:43:17.182380 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:17.182395 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:17.182429 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:43:17.183073 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:17.183088 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:17.183342 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:17.183616 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:17.184437 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:17.184442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:17.184453 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:17.184462 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:17.184515 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:17.184533 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:17.184649 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:17.184805 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 22:43:17.184976 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":128,"threads_started":1}
I20260810 22:43:17.187160 10568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:17.187983 10568 log.cc:824] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:17.188280 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:17.190251 10568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84: No bootstrap required, opened a new log
I20260810 22:43:17.191967 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:17.190453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35905 (local address 127.0.0.1:35227)
0810 22:43:17.190584 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:17.190589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:17.190756 (+ 167us) server_negotiation.cc:407] Connection header received
0810 22:43:17.190833 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:17.190836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:17.190866 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:17.190948 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:17.191585 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:17.191589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:17.191592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:17.191603 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:17.191629 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:17.191642 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:17.191745 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:17.191793 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 22:43:17.191824 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:43:17.194591 10568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f86bb368f421589aababa27813d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35227 } }
I20260810 22:43:17.194857 10568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:17.194913 10568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404f86bb368f421589aababa27813d84, State: Initialized, Role: FOLLOWER
I20260810 22:43:17.195261 10568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f86bb368f421589aababa27813d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35227 } }
I20260810 22:43:17.195369 10568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:17.195411 10568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:17.195477 10568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:17.196241 10568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f86bb368f421589aababa27813d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35227 } }
I20260810 22:43:17.196517 10568 leader_election.cc:302] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 404f86bb368f421589aababa27813d84; no voters:
I20260810 22:43:17.196878 10568 leader_election.cc:288] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:17.196964 10580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:17.197201 10580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [term 1 LEADER]: Becoming Leader. State: Replica: 404f86bb368f421589aababa27813d84, State: Running, Role: LEADER
I20260810 22:43:17.197553 10580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f86bb368f421589aababa27813d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35227 } }
I20260810 22:43:17.198081 10568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:17.199147 10583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 404f86bb368f421589aababa27813d84. Latest consensus state: current_term: 1 leader_uuid: "404f86bb368f421589aababa27813d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f86bb368f421589aababa27813d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35227 } } }
I20260810 22:43:17.199285 10583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:17.199570 10581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "404f86bb368f421589aababa27813d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f86bb368f421589aababa27813d84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35227 } } }
I20260810 22:43:17.199656 10581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:17.199687 10590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:17.202752 10590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:17.207906 10590 catalog_manager.cc:1347] Generated new cluster ID: 5eb2559a24b04a32b6284e115a59a598
I20260810 22:43:17.207973 10590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:17.216104 10590 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:17.216398 10590 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:17.223733 10590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 404f86bb368f421589aababa27813d84: Generated new TSK 0
I20260810 22:43:17.224421 10590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:17.234193 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 10489
2026-08-10T22:43:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:17 dist-test-slave-1x4g krb5kdc[10603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:17 dist-test-slave-1x4g krb5kdc[10603](info): set up 2 sockets
Aug 10 22:43:17 dist-test-slave-1x4g krb5kdc[10603](info): commencing operation
krb5kdc: starting...
W20260810 22:43:18.320297 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:18 dist-test-slave-1x4g krb5kdc[10603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:18.449586 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34263
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:18.560967 10619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:18.561259 10619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:18.561334 10619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:18.565181 10619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:18.565264 10619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:18.565302 10619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:18.565356 10619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:18.565395 10619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:18.570497 10619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:18.571578 10619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:18.572614 10619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:18.581241 10625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:18.581271 10624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:18.581300 10627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:18.581326 10619 server_base.cc:1034] running on GCE node
I20260810 22:43:18.582062 10619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:18.583084 10619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:18.584272 10619 hybrid_clock.cc:648] HybridClock initialized: now 1786401798584243 us; error 40 us; skew 500 ppm
Aug 10 22:43:18 dist-test-slave-1x4g krb5kdc[10603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:18.586958 10619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:18.587796 10619 webserver.cc:460] Webserver started at http://127.0.0.1:39207/ using document root <none> and password file <none>
I20260810 22:43:18.588336 10619 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:18.588433 10619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:18.588721 10619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:18.590737 10619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "a2257677a9334909b0b8ae55f18100db"
format_stamp: "Formatted at 2026-08-10 22:43:18 on dist-test-slave-1x4g"
server_key: "e8abba1f3125fc1fc79015e8ac2a59f5"
server_key_iv: "c724b742314ddac3bbaca2f70222b264"
server_key_version: "encryptionkey@0"
I20260810 22:43:18.591311 10619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "a2257677a9334909b0b8ae55f18100db"
format_stamp: "Formatted at 2026-08-10 22:43:18 on dist-test-slave-1x4g"
server_key: "e8abba1f3125fc1fc79015e8ac2a59f5"
server_key_iv: "c724b742314ddac3bbaca2f70222b264"
server_key_version: "encryptionkey@0"
I20260810 22:43:18.594983 10619 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260810 22:43:18.597393 10634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:18.598604 10619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:18.598728 10619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "a2257677a9334909b0b8ae55f18100db"
format_stamp: "Formatted at 2026-08-10 22:43:18 on dist-test-slave-1x4g"
server_key: "e8abba1f3125fc1fc79015e8ac2a59f5"
server_key_iv: "c724b742314ddac3bbaca2f70222b264"
server_key_version: "encryptionkey@0"
I20260810 22:43:18.598827 10619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:18.614405 10619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:18.617336 10619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:18.617538 10619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:18.624630 10619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34263
I20260810 22:43:18.624646 10699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34263 every 8 connection(s)
I20260810 22:43:18.625684 10619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:18.628376 10700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:18.633826 10700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db: Bootstrap starting.
I20260810 22:43:18.635771 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 10619
I20260810 22:43:18.635905 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:18.636206 20265 external_mini_cluster.cc:1428] Setting key c28190351b0fd635edba3fc2860073df
I20260810 22:43:18.636389 10700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:18.637396 10700 log.cc:824] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:18.639405 10700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db: No bootstrap required, opened a new log
I20260810 22:43:18.643702 10700 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2257677a9334909b0b8ae55f18100db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34263 } }
I20260810 22:43:18.644021 10700 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:18.644116 10700 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2257677a9334909b0b8ae55f18100db, State: Initialized, Role: FOLLOWER
I20260810 22:43:18.644639 10700 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2257677a9334909b0b8ae55f18100db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34263 } }
I20260810 22:43:18.644832 10700 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:18.644920 10700 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:18.645040 10700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 0 FOLLOWER]: Advancing to term 1
Aug 10 22:43:18 dist-test-slave-1x4g krb5kdc[10603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 22:43:18.645952 10700 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2257677a9334909b0b8ae55f18100db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34263 } }
I20260810 22:43:18.646276 10700 leader_election.cc:302] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2257677a9334909b0b8ae55f18100db; no voters:
I20260810 22:43:18.646610 10700 leader_election.cc:288] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:18.646675 10705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:18.646880 10705 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [term 1 LEADER]: Becoming Leader. State: Replica: a2257677a9334909b0b8ae55f18100db, State: Running, Role: LEADER
I20260810 22:43:18.647213 10705 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2257677a9334909b0b8ae55f18100db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34263 } }
I20260810 22:43:18.647867 10700 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:18.648677 10706 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2257677a9334909b0b8ae55f18100db. Latest consensus state: current_term: 1 leader_uuid: "a2257677a9334909b0b8ae55f18100db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2257677a9334909b0b8ae55f18100db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34263 } } }
I20260810 22:43:18.649050 10706 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:18.648941 10705 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2257677a9334909b0b8ae55f18100db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2257677a9334909b0b8ae55f18100db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34263 } } }
I20260810 22:43:18.649362 10705 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:18.649446 10716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:18.651691 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:18.637361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44139 (local address 127.0.0.1:34263)
0810 22:43:18.637785 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:18.637801 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:18.637834 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:43:18.638496 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:18.638513 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:18.638819 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:18.639089 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:18.640022 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:18.641040 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:18.642300 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:18.642751 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:18.642870 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:18.645852 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:18.645858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:18.645879 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:18.645904 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:18.647895 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:18.648366 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:18.648371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:18.648378 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:18.648516 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:18.648865 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:18.648872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:18.648877 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:18.649427 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:18.649656 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:18.650438 (+ 782us) server_negotiation.cc:299] Negotiation successful
0810 22:43:18.650746 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":136,"threads_started":1}
I20260810 22:43:18.653761 10716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:18.654884 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:18.661787 10716 catalog_manager.cc:1347] Generated new cluster ID: 34625b0a62f748a1bdd46b302d043388
I20260810 22:43:18.661847 10716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:18.670045 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:18.661190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52671 (local address 127.0.0.1:34263)
0810 22:43:18.661297 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:18.661301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:18.661572 (+ 271us) server_negotiation.cc:407] Connection header received
0810 22:43:18.661779 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:18.661783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:18.661833 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:18.661888 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:18.662808 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:18.663621 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:18.664716 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:18.665117 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:18.665223 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:18.666665 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:18.666669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:18.666672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:18.666698 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:18.668315 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:18.668738 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:18.668741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:18.668743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:18.668851 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:18.669110 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:18.669113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:18.669115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:18.669308 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:18.669444 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:18.669776 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 22:43:18.669884 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:43:18.684337 10716 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:18.685565 10716 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:18.690970 10716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2257677a9334909b0b8ae55f18100db: Generated new TSK 0
I20260810 22:43:18.691743 10716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:18.704346 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 10619
2026-08-10T22:43:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:18 dist-test-slave-1x4g krb5kdc[10733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:18 dist-test-slave-1x4g krb5kdc[10733](info): set up 2 sockets
Aug 10 22:43:18 dist-test-slave-1x4g krb5kdc[10733](info): commencing operation
krb5kdc: starting...
W20260810 22:43:19.782907 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:19 dist-test-slave-1x4g krb5kdc[10733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:19.903965 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:20.013995 10749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:20.014292 10749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:20.014376 10749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:20.018476 10749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:20.018565 10749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:20.018605 10749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:20.018656 10749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:20.018697 10749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:20.024101 10749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:20.025197 10749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:20.026219 10749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:20.034940 10759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:20.034940 10757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:20.034987 10755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:20.035303 10749 server_base.cc:1034] running on GCE node
I20260810 22:43:20.035866 10749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:20.036983 10749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:20.038195 10749 hybrid_clock.cc:648] HybridClock initialized: now 1786401800038175 us; error 31 us; skew 500 ppm
Aug 10 22:43:20 dist-test-slave-1x4g krb5kdc[10733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401800, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:20.041114 10749 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:20.041973 10749 webserver.cc:460] Webserver started at http://127.0.0.1:43551/ using document root <none> and password file <none>
I20260810 22:43:20.042522 10749 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:20.042614 10749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:20.042846 10749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:20.044876 10749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "aaa24c20c9334ac081f9dcbeee961e95"
format_stamp: "Formatted at 2026-08-10 22:43:20 on dist-test-slave-1x4g"
server_key: "b602519f7ba984596a62c105b80f26e5"
server_key_iv: "0b7a6234a9fe29bea72ae1f1ddc2bb8a"
server_key_version: "encryptionkey@0"
I20260810 22:43:20.045426 10749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "aaa24c20c9334ac081f9dcbeee961e95"
format_stamp: "Formatted at 2026-08-10 22:43:20 on dist-test-slave-1x4g"
server_key: "b602519f7ba984596a62c105b80f26e5"
server_key_iv: "0b7a6234a9fe29bea72ae1f1ddc2bb8a"
server_key_version: "encryptionkey@0"
I20260810 22:43:20.049003 10749 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 22:43:20.051353 10767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:20.052596 10749 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:20.052747 10749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "aaa24c20c9334ac081f9dcbeee961e95"
format_stamp: "Formatted at 2026-08-10 22:43:20 on dist-test-slave-1x4g"
server_key: "b602519f7ba984596a62c105b80f26e5"
server_key_iv: "0b7a6234a9fe29bea72ae1f1ddc2bb8a"
server_key_version: "encryptionkey@0"
I20260810 22:43:20.052908 10749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:20.089807 10749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:20.093127 10749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:20.093374 10749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:20.100837 10749 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40881
I20260810 22:43:20.100870 10825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40881 every 8 connection(s)
I20260810 22:43:20.101828 10749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:20.104619 10826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:20.110174 10826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95: Bootstrap starting.
I20260810 22:43:20.110559 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 10749
I20260810 22:43:20.110724 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:20.111002 20265 external_mini_cluster.cc:1428] Setting key 9c287bb55183ae734048eb2f92250ccf
I20260810 22:43:20.112938 10826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:20.113708 10826 log.cc:824] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:20.115547 10826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95: No bootstrap required, opened a new log
I20260810 22:43:20.119220 10826 raft_consensus.cc:348] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa24c20c9334ac081f9dcbeee961e95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260810 22:43:20.119561 10826 raft_consensus.cc:374] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:20.119649 10826 raft_consensus.cc:729] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa24c20c9334ac081f9dcbeee961e95, State: Initialized, Role: FOLLOWER
I20260810 22:43:20.120110 10826 consensus_queue.cc:260] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa24c20c9334ac081f9dcbeee961e95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260810 22:43:20.120219 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:20.112370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45165 (local address 127.0.0.1:40881)
0810 22:43:20.112894 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:20.112913 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:20.112950 (+ 37us) server_negotiation.cc:407] Connection header received
0810 22:43:20.113633 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:20.113650 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:20.114048 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:20.114377 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:20.114813 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:20.115828 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:20.116751 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:20.117275 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:20.117458 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:20.118413 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:20.118418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:20.118432 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:20.118831 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:20.118884 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:20.118910 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:20.119020 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:20.119175 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 22:43:20.119374 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":143,"threads_started":1}
I20260810 22:43:20.120257 10826 raft_consensus.cc:388] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:20.120364 10826 raft_consensus.cc:482] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:20.120484 10826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:20.121577 10826 raft_consensus.cc:504] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa24c20c9334ac081f9dcbeee961e95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260810 22:43:20.121917 10826 leader_election.cc:302] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aaa24c20c9334ac081f9dcbeee961e95; no voters:
I20260810 22:43:20.122229 10826 leader_election.cc:288] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:20.122303 10831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:20.122526 10831 raft_consensus.cc:686] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [term 1 LEADER]: Becoming Leader. State: Replica: aaa24c20c9334ac081f9dcbeee961e95, State: Running, Role: LEADER
I20260810 22:43:20.122839 10831 consensus_queue.cc:237] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa24c20c9334ac081f9dcbeee961e95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260810 22:43:20.123209 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:20.123523 10826 sys_catalog.cc:564] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:20.124362 10832 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aaa24c20c9334ac081f9dcbeee961e95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa24c20c9334ac081f9dcbeee961e95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260810 22:43:20.124516 10832 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:20.124871 10834 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aaa24c20c9334ac081f9dcbeee961e95. Latest consensus state: current_term: 1 leader_uuid: "aaa24c20c9334ac081f9dcbeee961e95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa24c20c9334ac081f9dcbeee961e95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260810 22:43:20.124969 10834 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95 [sys.catalog]: This master's current role is: LEADER
W20260810 22:43:20.129016 10855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:20.129112 10855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:20.129184 10847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:20.130033 10847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:20.130743 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:20.125500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55271 (local address 127.0.0.1:40881)
0810 22:43:20.125731 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:20.125735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:20.126156 (+ 421us) server_negotiation.cc:407] Connection header received
0810 22:43:20.126340 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:20.126343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:20.126405 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:20.126500 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:20.126906 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:20.127641 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:20.128766 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:20.129118 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:20.129226 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:20.130282 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:20.130286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:20.130289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:20.130303 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:20.130332 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:20.130346 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:20.130437 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:20.130557 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 22:43:20.130611 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260810 22:43:20.135835 10847 catalog_manager.cc:1347] Generated new cluster ID: 9c4c6cfb4dbc4f10bcb213d5f4ae3623
I20260810 22:43:20.135898 10847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:20.149472 10847 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:20.150373 10847 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:20.154713 10847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aaa24c20c9334ac081f9dcbeee961e95: Generated new TSK 0
I20260810 22:43:20.155332 10847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:20.160228 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 10749
2026-08-10T22:43:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:20 dist-test-slave-1x4g krb5kdc[10861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:20 dist-test-slave-1x4g krb5kdc[10861](info): set up 2 sockets
Aug 10 22:43:20 dist-test-slave-1x4g krb5kdc[10861](info): commencing operation
krb5kdc: starting...
W20260810 22:43:21.262049 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:21 dist-test-slave-1x4g krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:21.385283 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:21.498216 10877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:21.498574 10877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:21.498643 10877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:21.502631 10877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:21.502682 10877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:21.502701 10877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:21.502717 10877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:21.502732 10877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:21.507750 10877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:21.508702 10877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:21.509770 10877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:43:21.518304 10877 server_base.cc:1034] running on GCE node
W20260810 22:43:21.518252 10883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:21.518251 10886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:21.518276 10884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:21.519217 10877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:21.520556 10877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:21.521755 10877 hybrid_clock.cc:648] HybridClock initialized: now 1786401801521743 us; error 28 us; skew 500 ppm
Aug 10 22:43:21 dist-test-slave-1x4g krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401801, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:21.524562 10877 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:21.525487 10877 webserver.cc:460] Webserver started at http://127.0.0.1:43635/ using document root <none> and password file <none>
I20260810 22:43:21.526032 10877 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:21.526094 10877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:21.526305 10877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:21.528303 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "2ebb05f27ed7480b811d77278c47345e"
format_stamp: "Formatted at 2026-08-10 22:43:21 on dist-test-slave-1x4g"
server_key: "24fd7ce10bd7aa8d433837511f039b1b"
server_key_iv: "bce79e52c879e251a266cbecfccf1ff4"
server_key_version: "encryptionkey@0"
I20260810 22:43:21.528934 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "2ebb05f27ed7480b811d77278c47345e"
format_stamp: "Formatted at 2026-08-10 22:43:21 on dist-test-slave-1x4g"
server_key: "24fd7ce10bd7aa8d433837511f039b1b"
server_key_iv: "bce79e52c879e251a266cbecfccf1ff4"
server_key_version: "encryptionkey@0"
I20260810 22:43:21.532649 10877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:43:21.534965 10893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:21.536144 10877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:21.536242 10877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "2ebb05f27ed7480b811d77278c47345e"
format_stamp: "Formatted at 2026-08-10 22:43:21 on dist-test-slave-1x4g"
server_key: "24fd7ce10bd7aa8d433837511f039b1b"
server_key_iv: "bce79e52c879e251a266cbecfccf1ff4"
server_key_version: "encryptionkey@0"
I20260810 22:43:21.536374 10877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:21.541206 10877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:21.544056 10877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:21.544246 10877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:21.551275 10877 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41421
I20260810 22:43:21.551302 10950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41421 every 8 connection(s)
I20260810 22:43:21.552177 10877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:21.554934 10951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:21.560367 10951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e: Bootstrap starting.
I20260810 22:43:21.561322 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 10877
I20260810 22:43:21.561486 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:21.561779 20265 external_mini_cluster.cc:1428] Setting key 0ed756cb21fd80a769121d7b3529b131
I20260810 22:43:21.563174 10951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:21.564095 10951 log.cc:824] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:21.566382 10951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e: No bootstrap required, opened a new log
I20260810 22:43:21.567149 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:21.562849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40757 (local address 127.0.0.1:41421)
0810 22:43:21.563360 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:21.563394 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:21.563440 (+ 46us) server_negotiation.cc:407] Connection header received
0810 22:43:21.564335 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:21.564358 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:21.564674 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:21.565057 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:21.565778 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:21.565783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:21.565797 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:21.565809 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:21.565868 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:21.565895 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:21.566014 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:21.566144 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 22:43:21.566288 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":147,"threads_started":1}
I20260810 22:43:21.570161 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:21.570669 10951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb05f27ed7480b811d77278c47345e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41421 } }
I20260810 22:43:21.570873 10951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:21.570930 10951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebb05f27ed7480b811d77278c47345e, State: Initialized, Role: FOLLOWER
I20260810 22:43:21.571379 10951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb05f27ed7480b811d77278c47345e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41421 } }
I20260810 22:43:21.571548 10951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:21.571630 10951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:21.571758 10951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:21.572832 10951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb05f27ed7480b811d77278c47345e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41421 } }
I20260810 22:43:21.573170 10951 leader_election.cc:302] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ebb05f27ed7480b811d77278c47345e; no voters:
I20260810 22:43:21.573516 10951 leader_election.cc:288] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:21.573652 10961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:21.573874 10961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [term 1 LEADER]: Becoming Leader. State: Replica: 2ebb05f27ed7480b811d77278c47345e, State: Running, Role: LEADER
I20260810 22:43:21.574172 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:21.572440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37457 (local address 127.0.0.1:41421)
0810 22:43:21.572551 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:21.572556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:21.572708 (+ 152us) server_negotiation.cc:407] Connection header received
0810 22:43:21.572796 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:21.572799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:21.572835 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:21.572930 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:21.573713 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:21.573716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:21.573721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:21.573737 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:21.573768 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:21.573785 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:21.573843 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:21.574033 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 22:43:21.574070 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:43:21.574168 10961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb05f27ed7480b811d77278c47345e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41421 } }
I20260810 22:43:21.574851 10951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:21.575577 10963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ebb05f27ed7480b811d77278c47345e. Latest consensus state: current_term: 1 leader_uuid: "2ebb05f27ed7480b811d77278c47345e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb05f27ed7480b811d77278c47345e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41421 } } }
I20260810 22:43:21.575699 10963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:21.575996 10962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ebb05f27ed7480b811d77278c47345e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb05f27ed7480b811d77278c47345e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41421 } } }
I20260810 22:43:21.576086 10962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e [sys.catalog]: This master's current role is: LEADER
W20260810 22:43:21.579712 10981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:21.579778 10981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:21.579836 10969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:21.580647 10969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:21.586133 10969 catalog_manager.cc:1347] Generated new cluster ID: 01cd5ea4a78f4a1298981142672c74e2
I20260810 22:43:21.586194 10969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:21.602773 10969 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:21.603132 10969 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:21.607286 10969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ebb05f27ed7480b811d77278c47345e: Generated new TSK 0
I20260810 22:43:21.608021 10969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:21.615415 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 10877
2026-08-10T22:43:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:21 dist-test-slave-1x4g krb5kdc[10986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:21 dist-test-slave-1x4g krb5kdc[10986](info): set up 2 sockets
Aug 10 22:43:21 dist-test-slave-1x4g krb5kdc[10986](info): commencing operation
krb5kdc: starting...
W20260810 22:43:22.704353 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:22 dist-test-slave-1x4g krb5kdc[10986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:22.830240 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36435
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:22.940867 11001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:22.941114 11001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:22.941155 11001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:22.945037 11001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:22.945086 11001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:22.945106 11001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:22.945122 11001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:22.945137 11001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:22.950183 11001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:22.951138 11001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:22.952039 11001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:22.960283 11008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:22.960373 11009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:22.960479 11001 server_base.cc:1034] running on GCE node
W20260810 22:43:22.960309 11011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:22.961110 11001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:22.962194 11001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:22.963388 11001 hybrid_clock.cc:648] HybridClock initialized: now 1786401802963369 us; error 43 us; skew 500 ppm
Aug 10 22:43:22 dist-test-slave-1x4g krb5kdc[10986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401802, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:22.971207 11001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:22.972154 11001 webserver.cc:460] Webserver started at http://127.0.0.1:39965/ using document root <none> and password file <none>
I20260810 22:43:22.972736 11001 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:22.972854 11001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:22.973100 11001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:22.975246 11001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "9ab6ab38863d408d8346a19c73e7e188"
format_stamp: "Formatted at 2026-08-10 22:43:22 on dist-test-slave-1x4g"
server_key: "98b2775bcb8bd09d76f3667756158f6f"
server_key_iv: "3f39c6b587af6d3ca64294dc3a203dd7"
server_key_version: "encryptionkey@0"
I20260810 22:43:22.975847 11001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "9ab6ab38863d408d8346a19c73e7e188"
format_stamp: "Formatted at 2026-08-10 22:43:22 on dist-test-slave-1x4g"
server_key: "98b2775bcb8bd09d76f3667756158f6f"
server_key_iv: "3f39c6b587af6d3ca64294dc3a203dd7"
server_key_version: "encryptionkey@0"
I20260810 22:43:22.979822 11001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 22:43:22.982298 11021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:22.983493 11001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:22.983633 11001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "9ab6ab38863d408d8346a19c73e7e188"
format_stamp: "Formatted at 2026-08-10 22:43:22 on dist-test-slave-1x4g"
server_key: "98b2775bcb8bd09d76f3667756158f6f"
server_key_iv: "3f39c6b587af6d3ca64294dc3a203dd7"
server_key_version: "encryptionkey@0"
I20260810 22:43:22.983743 11001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:23.009910 11001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:23.013550 11001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:23.013765 11001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:23.020948 11001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36435
I20260810 22:43:23.021041 11082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36435 every 8 connection(s)
I20260810 22:43:23.021952 11001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:23.024693 11083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:23.027626 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11001
I20260810 22:43:23.027783 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:23.028057 20265 external_mini_cluster.cc:1428] Setting key b2985d71e1a1fab75cd94c5d7c3fa545
I20260810 22:43:23.030577 11083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188: Bootstrap starting.
I20260810 22:43:23.033255 11083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:23.034070 11083 log.cc:824] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:23.036047 11083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188: No bootstrap required, opened a new log
Aug 10 22:43:23 dist-test-slave-1x4g krb5kdc[10986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 22:43:23.039860 11083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab6ab38863d408d8346a19c73e7e188" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36435 } }
I20260810 22:43:23.040079 11083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:23.040148 11083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab6ab38863d408d8346a19c73e7e188, State: Initialized, Role: FOLLOWER
I20260810 22:43:23.040606 11083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab6ab38863d408d8346a19c73e7e188" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36435 } }
I20260810 22:43:23.040812 11083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:23.040892 11083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:23.041035 11083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:23.042064 11083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab6ab38863d408d8346a19c73e7e188" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36435 } }
I20260810 22:43:23.042464 11083 leader_election.cc:302] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ab6ab38863d408d8346a19c73e7e188; no voters:
I20260810 22:43:23.042816 11083 leader_election.cc:288] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:23.042889 11089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:23.043105 11089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab6ab38863d408d8346a19c73e7e188, State: Running, Role: LEADER
I20260810 22:43:23.043454 11089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab6ab38863d408d8346a19c73e7e188" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36435 } }
I20260810 22:43:23.043793 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:23.029461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:46925 (local address 127.0.0.1:36435)
0810 22:43:23.029841 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:23.029860 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:23.029901 (+ 41us) server_negotiation.cc:407] Connection header received
0810 22:43:23.030684 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:23.030706 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:23.031130 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:23.031484 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:23.032132 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:23.033645 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:23.034538 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:23.034987 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:23.035196 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:23.038089 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:23.038094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:23.038107 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:23.038132 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:23.040484 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:23.041130 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:23.041139 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:23.041147 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:23.041275 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:23.041638 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:23.041641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:23.041642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:23.042025 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:23.042210 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:23.042530 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 22:43:23.042750 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
I20260810 22:43:23.044152 11083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:23.045140 11092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ab6ab38863d408d8346a19c73e7e188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab6ab38863d408d8346a19c73e7e188" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36435 } } }
I20260810 22:43:23.045284 11092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:23.045629 11100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:23.045991 11091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ab6ab38863d408d8346a19c73e7e188. Latest consensus state: current_term: 1 leader_uuid: "9ab6ab38863d408d8346a19c73e7e188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab6ab38863d408d8346a19c73e7e188" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36435 } } }
I20260810 22:43:23.046084 11091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:23.046834 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:23.049015 11100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:23.054857 11100 catalog_manager.cc:1347] Generated new cluster ID: 1087232bf87d40119f1f4c1fe716adba
I20260810 22:43:23.054921 11100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:23.061648 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:23.049040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:34203 (local address 127.0.0.1:36435)
0810 22:43:23.049132 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:23.049136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:23.053085 (+ 3949us) server_negotiation.cc:407] Connection header received
0810 22:43:23.053260 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:23.053263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:23.053324 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:23.053415 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:23.054124 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:23.054723 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:23.055866 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:23.056249 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:23.056366 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:23.058002 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:23.058006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:23.058009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:23.058035 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:23.059628 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:23.060136 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:23.060142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:23.060146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:23.060251 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:23.060580 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:23.060584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:23.060585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:23.060830 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:23.061010 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:23.061233 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 22:43:23.061500 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:43:23.080374 11100 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:23.081423 11100 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:23.087144 11100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ab6ab38863d408d8346a19c73e7e188: Generated new TSK 0
I20260810 22:43:23.087864 11100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:23.095120 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11001
W20260810 22:43:23.104322 10984 connection.cc:535] client connection to 127.0.0.1:36435 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36435): Connection reset by peer (error 104)
2026-08-10T22:43:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:23 dist-test-slave-1x4g krb5kdc[11122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:23 dist-test-slave-1x4g krb5kdc[11122](info): set up 2 sockets
Aug 10 22:43:23 dist-test-slave-1x4g krb5kdc[11122](info): commencing operation
krb5kdc: starting...
W20260810 22:43:24.169953 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:24 dist-test-slave-1x4g krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:24.293401 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:24.405305 11137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:24.405557 11137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:24.405598 11137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:24.409533 11137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:24.409590 11137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:24.409610 11137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:24.409626 11137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:24.409641 11137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:24.414733 11137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:24.415709 11137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:24.416689 11137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:24.425930 11145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:24.425974 11142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:24.425941 11143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:24.426120 11137 server_base.cc:1034] running on GCE node
I20260810 22:43:24.426846 11137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:24.427963 11137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:24.429145 11137 hybrid_clock.cc:648] HybridClock initialized: now 1786401804429134 us; error 28 us; skew 500 ppm
Aug 10 22:43:24 dist-test-slave-1x4g krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401804, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:24.431913 11137 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:24.432840 11137 webserver.cc:460] Webserver started at http://127.0.0.1:46555/ using document root <none> and password file <none>
I20260810 22:43:24.433398 11137 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:24.433486 11137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:24.433712 11137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:24.435715 11137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "507efe6070f34e0b93b74a6732417f7e"
format_stamp: "Formatted at 2026-08-10 22:43:24 on dist-test-slave-1x4g"
server_key: "c8becb029769bb6e437ce6219f283fde"
server_key_iv: "c795c5249f28a5bb93ca3618708c14bf"
server_key_version: "encryptionkey@0"
I20260810 22:43:24.436290 11137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "507efe6070f34e0b93b74a6732417f7e"
format_stamp: "Formatted at 2026-08-10 22:43:24 on dist-test-slave-1x4g"
server_key: "c8becb029769bb6e437ce6219f283fde"
server_key_iv: "c795c5249f28a5bb93ca3618708c14bf"
server_key_version: "encryptionkey@0"
I20260810 22:43:24.439951 11137 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 22:43:24.442314 11154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:24.443499 11137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:24.443655 11137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "507efe6070f34e0b93b74a6732417f7e"
format_stamp: "Formatted at 2026-08-10 22:43:24 on dist-test-slave-1x4g"
server_key: "c8becb029769bb6e437ce6219f283fde"
server_key_iv: "c795c5249f28a5bb93ca3618708c14bf"
server_key_version: "encryptionkey@0"
I20260810 22:43:24.443760 11137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:24.455572 11137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:24.458272 11137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:24.458460 11137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:24.466003 11137 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36965
I20260810 22:43:24.466027 11217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36965 every 8 connection(s)
I20260810 22:43:24.466894 11137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:24.469049 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11137
I20260810 22:43:24.469192 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:24.469530 20265 external_mini_cluster.cc:1428] Setting key e294e128bd4391446956cc0bb50215f4
I20260810 22:43:24.469732 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:24.475865 11218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e: Bootstrap starting.
I20260810 22:43:24.478088 11218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:24.478632 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:24.470971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:38589 (local address 127.0.0.1:36965)
0810 22:43:24.471409 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:24.471424 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:24.471458 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:43:24.472410 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:24.472428 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:24.472744 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:24.473118 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:24.473515 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:24.474443 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:24.475360 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:24.475861 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:24.476033 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:24.476988 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:24.476993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:24.477005 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:24.477364 (+ 359us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:24.477424 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:24.477446 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:24.477585 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:24.477769 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 22:43:24.477906 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":143,"threads_started":1}
I20260810 22:43:24.478868 11218 log.cc:824] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:24.480741 11218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e: No bootstrap required, opened a new log
I20260810 22:43:24.481494 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:24.485100 11218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507efe6070f34e0b93b74a6732417f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260810 22:43:24.485364 11218 raft_consensus.cc:374] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:24.485423 11218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 507efe6070f34e0b93b74a6732417f7e, State: Initialized, Role: FOLLOWER
I20260810 22:43:24.485857 11218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507efe6070f34e0b93b74a6732417f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260810 22:43:24.486045 11218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:24.486115 11218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:24.486209 11218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:24.487339 11218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507efe6070f34e0b93b74a6732417f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260810 22:43:24.487622 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:24.483538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:44805 (local address 127.0.0.1:36965)
0810 22:43:24.483642 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:24.483645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:24.483846 (+ 201us) server_negotiation.cc:407] Connection header received
0810 22:43:24.483960 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:24.483962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:24.483997 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:24.484062 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:24.484370 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:24.484895 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:24.485944 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:24.486274 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:24.486374 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:24.487159 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:24.487162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:24.487164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:24.487174 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:24.487197 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:24.487206 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:24.487265 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:24.487495 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 22:43:24.487528 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 22:43:24.487700 11218 leader_election.cc:302] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 507efe6070f34e0b93b74a6732417f7e; no voters:
I20260810 22:43:24.488061 11218 leader_election.cc:288] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:24.488130 11228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:24.488336 11228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [term 1 LEADER]: Becoming Leader. State: Replica: 507efe6070f34e0b93b74a6732417f7e, State: Running, Role: LEADER
I20260810 22:43:24.488699 11228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507efe6070f34e0b93b74a6732417f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260810 22:43:24.489264 11218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:24.489789 11228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 507efe6070f34e0b93b74a6732417f7e. Latest consensus state: current_term: 1 leader_uuid: "507efe6070f34e0b93b74a6732417f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507efe6070f34e0b93b74a6732417f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } } }
I20260810 22:43:24.489917 11228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:24.490568 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "507efe6070f34e0b93b74a6732417f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507efe6070f34e0b93b74a6732417f7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } } }
I20260810 22:43:24.490692 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:24.490816 11237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:24.493768 11237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:24.498864 11237 catalog_manager.cc:1347] Generated new cluster ID: 0ef96133d59a41689f8ac974513a8935
I20260810 22:43:24.498927 11237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:24.508983 11237 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:24.509835 11237 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:24.520038 11237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 507efe6070f34e0b93b74a6732417f7e: Generated new TSK 0
I20260810 22:43:24.520677 11237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:24.524660 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11137
W20260810 22:43:24.532742 11119 connection.cc:535] client connection to 127.0.0.1:36965 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36965): Connection reset by peer (error 104)
2026-08-10T22:43:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:24 dist-test-slave-1x4g krb5kdc[11254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:24 dist-test-slave-1x4g krb5kdc[11254](info): set up 2 sockets
Aug 10 22:43:24 dist-test-slave-1x4g krb5kdc[11254](info): commencing operation
krb5kdc: starting...
W20260810 22:43:25.616330 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:25 dist-test-slave-1x4g krb5kdc[11254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:25.741250 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35517
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:25.852715 11269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:25.853034 11269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:25.853120 11269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:25.857133 11269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:25.857208 11269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:25.857245 11269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:25.857297 11269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:25.857336 11269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:25.862572 11269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:25.863667 11269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:25.864712 11269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:25.873154 11275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:25.873150 11278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:25.873131 11276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:25.873438 11269 server_base.cc:1034] running on GCE node
I20260810 22:43:25.873984 11269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:25.874989 11269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:25.876143 11269 hybrid_clock.cc:648] HybridClock initialized: now 1786401805876130 us; error 24 us; skew 500 ppm
Aug 10 22:43:25 dist-test-slave-1x4g krb5kdc[11254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401805, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:25.878926 11269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:25.879812 11269 webserver.cc:460] Webserver started at http://127.0.0.1:43151/ using document root <none> and password file <none>
I20260810 22:43:25.880363 11269 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:25.880450 11269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:25.880677 11269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:25.882721 11269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "e96ddbe96aaf4a1888108b776d9724a8"
format_stamp: "Formatted at 2026-08-10 22:43:25 on dist-test-slave-1x4g"
server_key: "511b1a82d8326a6a8cd981e98d5f9fe9"
server_key_iv: "007ccd12a68e9ead66149dc4054baa86"
server_key_version: "encryptionkey@0"
I20260810 22:43:25.883322 11269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "e96ddbe96aaf4a1888108b776d9724a8"
format_stamp: "Formatted at 2026-08-10 22:43:25 on dist-test-slave-1x4g"
server_key: "511b1a82d8326a6a8cd981e98d5f9fe9"
server_key_iv: "007ccd12a68e9ead66149dc4054baa86"
server_key_version: "encryptionkey@0"
I20260810 22:43:25.886956 11269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:43:25.889271 11286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:25.890442 11269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:25.890571 11269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "e96ddbe96aaf4a1888108b776d9724a8"
format_stamp: "Formatted at 2026-08-10 22:43:25 on dist-test-slave-1x4g"
server_key: "511b1a82d8326a6a8cd981e98d5f9fe9"
server_key_iv: "007ccd12a68e9ead66149dc4054baa86"
server_key_version: "encryptionkey@0"
I20260810 22:43:25.890673 11269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:25.895308 11269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:25.898450 11269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:25.898628 11269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:25.905405 11269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35517
I20260810 22:43:25.905447 11343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35517 every 8 connection(s)
I20260810 22:43:25.906312 11269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:25.907327 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11269
I20260810 22:43:25.907483 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:25.907799 20265 external_mini_cluster.cc:1428] Setting key 7b3130a8f2184040a6f3abc3a775b5c3
I20260810 22:43:25.909667 11344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:25.913364 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:25.909224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:56337 (local address 127.0.0.1:35517)
0810 22:43:25.909630 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:25.909649 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:25.909692 (+ 43us) server_negotiation.cc:407] Connection header received
0810 22:43:25.910588 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:25.910610 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:25.910948 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:25.911304 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:25.911973 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:25.911979 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:25.911993 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:25.912006 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:25.912063 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:25.912089 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:25.912229 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:25.912300 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 22:43:25.912471 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260810 22:43:25.916148 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:25.916687 11344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8: Bootstrap starting.
I20260810 22:43:25.919307 11344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:25.920025 11344 log.cc:824] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:25.920224 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:25.918511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:36441 (local address 127.0.0.1:35517)
0810 22:43:25.918632 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:25.918636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:25.918826 (+ 190us) server_negotiation.cc:407] Connection header received
0810 22:43:25.918982 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:25.918985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:25.919015 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:25.919088 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:25.919824 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:25.919828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:25.919830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:25.919841 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:25.919868 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:25.919883 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:25.919965 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:25.920048 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 22:43:25.920086 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:43:25.922041 11344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8: No bootstrap required, opened a new log
I20260810 22:43:25.925462 11344 raft_consensus.cc:348] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96ddbe96aaf4a1888108b776d9724a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35517 } }
I20260810 22:43:25.925698 11344 raft_consensus.cc:374] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:25.925774 11344 raft_consensus.cc:729] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96ddbe96aaf4a1888108b776d9724a8, State: Initialized, Role: FOLLOWER
I20260810 22:43:25.926195 11344 consensus_queue.cc:260] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96ddbe96aaf4a1888108b776d9724a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35517 } }
I20260810 22:43:25.926352 11344 raft_consensus.cc:388] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:25.926437 11344 raft_consensus.cc:482] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:25.926553 11344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:25.927577 11344 raft_consensus.cc:504] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96ddbe96aaf4a1888108b776d9724a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35517 } }
I20260810 22:43:25.927904 11344 leader_election.cc:302] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e96ddbe96aaf4a1888108b776d9724a8; no voters:
I20260810 22:43:25.928195 11344 leader_election.cc:288] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:25.928305 11357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:25.928545 11357 raft_consensus.cc:686] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [term 1 LEADER]: Becoming Leader. State: Replica: e96ddbe96aaf4a1888108b776d9724a8, State: Running, Role: LEADER
I20260810 22:43:25.928895 11357 consensus_queue.cc:237] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96ddbe96aaf4a1888108b776d9724a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35517 } }
I20260810 22:43:25.929394 11344 sys_catalog.cc:564] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:25.930660 11358 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e96ddbe96aaf4a1888108b776d9724a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96ddbe96aaf4a1888108b776d9724a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35517 } } }
I20260810 22:43:25.930688 11359 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e96ddbe96aaf4a1888108b776d9724a8. Latest consensus state: current_term: 1 leader_uuid: "e96ddbe96aaf4a1888108b776d9724a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96ddbe96aaf4a1888108b776d9724a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35517 } } }
I20260810 22:43:25.930789 11359 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:25.930788 11358 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:25.931965 11368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:25.934679 11368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:25.939603 11368 catalog_manager.cc:1347] Generated new cluster ID: a616f316a5c94d5198ef9d9b5f5bc114
I20260810 22:43:25.939662 11368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:25.953686 11368 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:25.954026 11368 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:25.965425 11368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e96ddbe96aaf4a1888108b776d9724a8: Generated new TSK 0
I20260810 22:43:25.966033 11368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:25.973747 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11269
2026-08-10T22:43:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:26 dist-test-slave-1x4g krb5kdc[11381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:26 dist-test-slave-1x4g krb5kdc[11381](info): set up 2 sockets
Aug 10 22:43:26 dist-test-slave-1x4g krb5kdc[11381](info): commencing operation
krb5kdc: starting...
W20260810 22:43:27.051041 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:27 dist-test-slave-1x4g krb5kdc[11381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:27.175474 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:27.286310 11396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:27.286586 11396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:27.286662 11396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:27.290692 11396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:27.290892 11396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:27.290982 11396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:27.291039 11396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:27.291091 11396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:27.296398 11396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35211
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:27.297466 11396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:27.298458 11396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:27.306694 11403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:27.306820 11396 server_base.cc:1034] running on GCE node
W20260810 22:43:27.306705 11401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:27.306704 11405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:27.307560 11396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:27.308633 11396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:27.309811 11396 hybrid_clock.cc:648] HybridClock initialized: now 1786401807309797 us; error 29 us; skew 500 ppm
Aug 10 22:43:27 dist-test-slave-1x4g krb5kdc[11381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401807, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:27.312621 11396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:27.313553 11396 webserver.cc:460] Webserver started at http://127.0.0.1:32779/ using document root <none> and password file <none>
I20260810 22:43:27.314101 11396 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:27.314188 11396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:27.314409 11396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:27.316382 11396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "98c0ff48b9334daaafae39ec0419525a"
format_stamp: "Formatted at 2026-08-10 22:43:27 on dist-test-slave-1x4g"
server_key: "b8f22a3cd9cfe6da071b3ff2469d83f9"
server_key_iv: "bc5ccf2dbc4d968db59e2d947563d0a5"
server_key_version: "encryptionkey@0"
I20260810 22:43:27.317015 11396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "98c0ff48b9334daaafae39ec0419525a"
format_stamp: "Formatted at 2026-08-10 22:43:27 on dist-test-slave-1x4g"
server_key: "b8f22a3cd9cfe6da071b3ff2469d83f9"
server_key_iv: "bc5ccf2dbc4d968db59e2d947563d0a5"
server_key_version: "encryptionkey@0"
I20260810 22:43:27.320554 11396 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260810 22:43:27.322985 11412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:27.324178 11396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:27.324311 11396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "98c0ff48b9334daaafae39ec0419525a"
format_stamp: "Formatted at 2026-08-10 22:43:27 on dist-test-slave-1x4g"
server_key: "b8f22a3cd9cfe6da071b3ff2469d83f9"
server_key_iv: "bc5ccf2dbc4d968db59e2d947563d0a5"
server_key_version: "encryptionkey@0"
I20260810 22:43:27.324415 11396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:27.368130 11396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:27.371500 11396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:27.371709 11396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:27.378852 11396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35211
I20260810 22:43:27.378881 11472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35211 every 8 connection(s)
I20260810 22:43:27.379767 11396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:27.382113 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11396
I20260810 22:43:27.382261 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:27.382608 20265 external_mini_cluster.cc:1428] Setting key 92d80016f3e5ccf02d3115d86cb7a9d3
I20260810 22:43:27.382820 11473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:27.389086 11473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a: Bootstrap starting.
I20260810 22:43:27.391345 11473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a: Neither blocks nor log segments found. Creating new log.
Aug 10 22:43:27 dist-test-slave-1x4g krb5kdc[11381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 22:43:27.392138 11473 log.cc:824] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:27.394002 11473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a: No bootstrap required, opened a new log
I20260810 22:43:27.396944 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:27.384029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:34601 (local address 127.0.0.1:35211)
0810 22:43:27.384464 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:27.384479 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:27.384513 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:43:27.385561 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:27.385580 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:27.385898 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:27.386214 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:27.387010 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:27.387957 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:27.388816 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:27.389331 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:27.389502 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:27.392193 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:27.392215 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:27.392234 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:27.392266 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:27.393983 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:27.394621 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:27.394625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:27.394629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:27.394732 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:27.395078 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:27.395081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:27.395082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:27.395437 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:27.395593 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:27.395928 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 22:43:27.396143 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":157,"threads_started":1}
I20260810 22:43:27.398101 11473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c0ff48b9334daaafae39ec0419525a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35211 } }
I20260810 22:43:27.398362 11473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:27.398427 11473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98c0ff48b9334daaafae39ec0419525a, State: Initialized, Role: FOLLOWER
I20260810 22:43:27.398872 11473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c0ff48b9334daaafae39ec0419525a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35211 } }
I20260810 22:43:27.399047 11473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:27.399119 11473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:27.399230 11473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:27.399938 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:27.400359 11473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c0ff48b9334daaafae39ec0419525a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35211 } }
I20260810 22:43:27.400738 11473 leader_election.cc:302] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98c0ff48b9334daaafae39ec0419525a; no voters:
I20260810 22:43:27.401150 11473 leader_election.cc:288] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:27.401248 11481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:27.401564 11481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [term 1 LEADER]: Becoming Leader. State: Replica: 98c0ff48b9334daaafae39ec0419525a, State: Running, Role: LEADER
I20260810 22:43:27.401908 11481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c0ff48b9334daaafae39ec0419525a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35211 } }
I20260810 22:43:27.402489 11473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:27.403638 11487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98c0ff48b9334daaafae39ec0419525a. Latest consensus state: current_term: 1 leader_uuid: "98c0ff48b9334daaafae39ec0419525a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c0ff48b9334daaafae39ec0419525a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35211 } } }
I20260810 22:43:27.403919 11485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98c0ff48b9334daaafae39ec0419525a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c0ff48b9334daaafae39ec0419525a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35211 } } }
I20260810 22:43:27.404065 11485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:27.404335 11487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:27.404951 11497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:27.408100 11497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:27.411245 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:27.402168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:43325 (local address 127.0.0.1:35211)
0810 22:43:27.402267 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:27.402272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:27.403318 (+ 1046us) server_negotiation.cc:407] Connection header received
0810 22:43:27.403503 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:27.403506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:27.403551 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:27.403604 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:27.404568 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:27.405364 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:27.406249 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:27.406482 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:27.406589 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:27.407911 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:27.407914 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:27.407917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:27.407940 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:27.409408 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:27.409873 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:27.409876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:27.409878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:27.409956 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:27.410330 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:27.410334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:27.410335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:27.410544 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:27.410684 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:27.411003 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 22:43:27.411100 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 22:43:27.414439 11497 catalog_manager.cc:1347] Generated new cluster ID: 5fcd64e4cf8c43c6a0058890bd2a6fde
I20260810 22:43:27.414505 11497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:27.433785 11497 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:27.434687 11497 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:27.449007 11497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98c0ff48b9334daaafae39ec0419525a: Generated new TSK 0
I20260810 22:43:27.449745 11497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:27.460764 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11396
W20260810 22:43:27.469034 11379 connection.cc:535] client connection to 127.0.0.1:35211 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35211): Connection reset by peer (error 104)
2026-08-10T22:43:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:27 dist-test-slave-1x4g krb5kdc[11510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:27 dist-test-slave-1x4g krb5kdc[11510](info): set up 2 sockets
Aug 10 22:43:27 dist-test-slave-1x4g krb5kdc[11510](info): commencing operation
krb5kdc: starting...
W20260810 22:43:28.550143 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:28 dist-test-slave-1x4g krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:28.676518 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:28.788159 11524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:28.788439 11524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:28.788517 11524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:28.792450 11524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:28.792543 11524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:28.792579 11524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:28.792637 11524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:28.792675 11524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:28.797796 11524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:28.798836 11524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:28.799831 11524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:43:28.808073 11524 server_base.cc:1034] running on GCE node
W20260810 22:43:28.808014 11530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:28.808033 11529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:28.808028 11532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:28.808892 11524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:28.809949 11524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:28.811103 11524 hybrid_clock.cc:648] HybridClock initialized: now 1786401808811084 us; error 30 us; skew 500 ppm
Aug 10 22:43:28 dist-test-slave-1x4g krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401808, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:28.817894 11524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:28.818742 11524 webserver.cc:460] Webserver started at http://127.0.0.1:38293/ using document root <none> and password file <none>
I20260810 22:43:28.819238 11524 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:28.819293 11524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:28.819453 11524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:28.821517 11524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "37166c412b13456b86ae5d3552508ade"
format_stamp: "Formatted at 2026-08-10 22:43:28 on dist-test-slave-1x4g"
server_key: "a4cd946573160fa0ee12aeab11fb37a3"
server_key_iv: "7735751e8c3ed8380af248ededa4b248"
server_key_version: "encryptionkey@0"
I20260810 22:43:28.822026 11524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "37166c412b13456b86ae5d3552508ade"
format_stamp: "Formatted at 2026-08-10 22:43:28 on dist-test-slave-1x4g"
server_key: "a4cd946573160fa0ee12aeab11fb37a3"
server_key_iv: "7735751e8c3ed8380af248ededa4b248"
server_key_version: "encryptionkey@0"
I20260810 22:43:28.825539 11524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:28.829121 11539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:28.830794 11524 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 22:43:28.830924 11524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "37166c412b13456b86ae5d3552508ade"
format_stamp: "Formatted at 2026-08-10 22:43:28 on dist-test-slave-1x4g"
server_key: "a4cd946573160fa0ee12aeab11fb37a3"
server_key_iv: "7735751e8c3ed8380af248ededa4b248"
server_key_version: "encryptionkey@0"
I20260810 22:43:28.831063 11524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:28.852105 11524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:28.855228 11524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:28.855422 11524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:28.862533 11524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35103
I20260810 22:43:28.862560 11602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35103 every 8 connection(s)
I20260810 22:43:28.863457 11524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:28.866178 11603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:28.871382 11603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade: Bootstrap starting.
I20260810 22:43:28.872912 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11524
I20260810 22:43:28.873060 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:28.873353 20265 external_mini_cluster.cc:1428] Setting key 8ee7be4f593c258ac43884813bd11d89
I20260810 22:43:28.874090 11603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:28.874987 11603 log.cc:824] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:28.876749 11603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade: No bootstrap required, opened a new log
I20260810 22:43:28.879949 11603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37166c412b13456b86ae5d3552508ade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260810 22:43:28.880182 11603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:28.880236 11603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37166c412b13456b86ae5d3552508ade, State: Initialized, Role: FOLLOWER
I20260810 22:43:28.880731 11603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37166c412b13456b86ae5d3552508ade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260810 22:43:28.880934 11603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:28.881016 11603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:28.881129 11603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:28.881989 11603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37166c412b13456b86ae5d3552508ade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260810 22:43:28.882222 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:28.874558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:44477 (local address 127.0.0.1:35103)
0810 22:43:28.874983 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:28.874998 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:28.875042 (+ 44us) server_negotiation.cc:407] Connection header received
0810 22:43:28.875699 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:28.875714 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:28.876029 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:28.876345 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:28.876867 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:28.877855 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:28.878737 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:28.879217 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:28.879406 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:28.880283 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:28.880289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:28.880320 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:28.880824 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:28.880895 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:28.880936 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:28.881074 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:28.881323 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 22:43:28.881469 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":129,"threads_started":1}
I20260810 22:43:28.882319 11603 leader_election.cc:302] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37166c412b13456b86ae5d3552508ade; no voters:
I20260810 22:43:28.882645 11603 leader_election.cc:288] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:28.882735 11609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:28.882974 11609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [term 1 LEADER]: Becoming Leader. State: Replica: 37166c412b13456b86ae5d3552508ade, State: Running, Role: LEADER
I20260810 22:43:28.883258 11609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37166c412b13456b86ae5d3552508ade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260810 22:43:28.883846 11603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:28.884425 11609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37166c412b13456b86ae5d3552508ade. Latest consensus state: current_term: 1 leader_uuid: "37166c412b13456b86ae5d3552508ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37166c412b13456b86ae5d3552508ade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } } }
I20260810 22:43:28.884534 11609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:28.884879 11610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37166c412b13456b86ae5d3552508ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37166c412b13456b86ae5d3552508ade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } } }
I20260810 22:43:28.884943 11615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:28.884974 11610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:28.885635 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:28.888427 11615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:28.893195 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:28.887793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:52557 (local address 127.0.0.1:35103)
0810 22:43:28.887907 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:28.887911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:28.888397 (+ 486us) server_negotiation.cc:407] Connection header received
0810 22:43:28.888565 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:28.888567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:28.888607 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:28.888649 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:28.889090 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:28.889781 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:28.890973 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:28.891294 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:28.891410 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:28.892655 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:28.892658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:28.892661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:28.892673 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:28.892700 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:28.892718 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:28.892853 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:28.893046 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 22:43:28.893077 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:43:28.894274 11615 catalog_manager.cc:1347] Generated new cluster ID: de10d131f87d41e292fa6ae725846d65
I20260810 22:43:28.894348 11615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:28.908293 11615 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:28.909523 11615 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:28.914270 11615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37166c412b13456b86ae5d3552508ade: Generated new TSK 0
I20260810 22:43:28.915174 11615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:28.920632 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11524
W20260810 22:43:28.932646 11507 connection.cc:535] client connection to 127.0.0.1:35103 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35103): Connection reset by peer (error 104)
2026-08-10T22:43:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:28 dist-test-slave-1x4g krb5kdc[11642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:28 dist-test-slave-1x4g krb5kdc[11642](info): set up 2 sockets
Aug 10 22:43:28 dist-test-slave-1x4g krb5kdc[11642](info): commencing operation
krb5kdc: starting...
W20260810 22:43:29.999392 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:30 dist-test-slave-1x4g krb5kdc[11642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 22:43:30.127651 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:30.241741 11656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:30.242059 11656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:30.242138 11656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:30.246142 11656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:30.246222 11656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:30.246258 11656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:30.246308 11656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:30.246344 11656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:30.251560 11656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:30.252710 11656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:30.253805 11656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:30.262213 11663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:30.262247 11662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:30.262260 11665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:30.262452 11656 server_base.cc:1034] running on GCE node
I20260810 22:43:30.263113 11656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:30.264133 11656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:30.265312 11656 hybrid_clock.cc:648] HybridClock initialized: now 1786401810265277 us; error 34 us; skew 500 ppm
Aug 10 22:43:30 dist-test-slave-1x4g krb5kdc[11642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:30.268117 11656 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 22:43:30.269021 11656 webserver.cc:460] Webserver started at http://127.0.0.1:35223/ using document root <none> and password file <none>
I20260810 22:43:30.269563 11656 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:30.269624 11656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:30.269857 11656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:30.271850 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "5f83d2e3168149f3aa4fad148b2762c3"
format_stamp: "Formatted at 2026-08-10 22:43:30 on dist-test-slave-1x4g"
server_key: "d5165a8f45a5a3259e44e1b1b454ee7b"
server_key_iv: "eff242fac8d471716af0521dfcdec431"
server_key_version: "encryptionkey@0"
I20260810 22:43:30.272423 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "5f83d2e3168149f3aa4fad148b2762c3"
format_stamp: "Formatted at 2026-08-10 22:43:30 on dist-test-slave-1x4g"
server_key: "d5165a8f45a5a3259e44e1b1b454ee7b"
server_key_iv: "eff242fac8d471716af0521dfcdec431"
server_key_version: "encryptionkey@0"
I20260810 22:43:30.276010 11656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260810 22:43:30.278579 11672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:30.279794 11656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:30.279953 11656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "5f83d2e3168149f3aa4fad148b2762c3"
format_stamp: "Formatted at 2026-08-10 22:43:30 on dist-test-slave-1x4g"
server_key: "d5165a8f45a5a3259e44e1b1b454ee7b"
server_key_iv: "eff242fac8d471716af0521dfcdec431"
server_key_version: "encryptionkey@0"
I20260810 22:43:30.280066 11656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:30.284623 11656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:30.287523 11656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:30.287746 11656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:30.294780 11656 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33153
I20260810 22:43:30.294855 11728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33153 every 8 connection(s)
I20260810 22:43:30.295698 11656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:30.298563 11729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:30.303792 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11656
I20260810 22:43:30.303944 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:30.304210 20265 external_mini_cluster.cc:1428] Setting key ff3c70a56f8f890fb46ecb9b9e7ec451
I20260810 22:43:30.304698 11729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3: Bootstrap starting.
I20260810 22:43:30.307545 11729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:30.308508 11729 log.cc:824] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:30.310237 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:30.305703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:42857 (local address 127.0.0.1:33153)
0810 22:43:30.306140 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:30.306158 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:30.306200 (+ 42us) server_negotiation.cc:407] Connection header received
0810 22:43:30.307099 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:30.307122 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:30.307500 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:30.307916 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:30.308811 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:30.308819 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:30.308834 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:30.308847 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:30.308914 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:30.308936 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:30.309058 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:30.309166 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 22:43:30.309327 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":101,"threads_started":1}
I20260810 22:43:30.311580 11729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3: No bootstrap required, opened a new log
I20260810 22:43:30.313408 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:30.316388 11729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f83d2e3168149f3aa4fad148b2762c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33153 } }
I20260810 22:43:30.316741 11729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:30.316851 11729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f83d2e3168149f3aa4fad148b2762c3, State: Initialized, Role: FOLLOWER
I20260810 22:43:30.317350 11729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f83d2e3168149f3aa4fad148b2762c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33153 } }
I20260810 22:43:30.317533 11729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:30.317632 11729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:30.317772 11729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:30.317972 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:30.315944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:53065 (local address 127.0.0.1:33153)
0810 22:43:30.316066 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:30.316070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:30.316215 (+ 145us) server_negotiation.cc:407] Connection header received
0810 22:43:30.316305 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:30.316308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:30.316339 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:30.316452 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:30.317311 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:30.317314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:30.317317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:30.317329 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:30.317359 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:30.317384 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:30.317478 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:30.317762 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 22:43:30.317840 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:43:30.319094 11729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f83d2e3168149f3aa4fad148b2762c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33153 } }
I20260810 22:43:30.319455 11729 leader_election.cc:302] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f83d2e3168149f3aa4fad148b2762c3; no voters:
I20260810 22:43:30.319785 11729 leader_election.cc:288] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:30.319880 11744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:30.320119 11744 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [term 1 LEADER]: Becoming Leader. State: Replica: 5f83d2e3168149f3aa4fad148b2762c3, State: Running, Role: LEADER
I20260810 22:43:30.320462 11744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f83d2e3168149f3aa4fad148b2762c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33153 } }
I20260810 22:43:30.321146 11729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:30.322451 11745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f83d2e3168149f3aa4fad148b2762c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f83d2e3168149f3aa4fad148b2762c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33153 } } }
I20260810 22:43:30.322576 11745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:30.323649 11746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f83d2e3168149f3aa4fad148b2762c3. Latest consensus state: current_term: 1 leader_uuid: "5f83d2e3168149f3aa4fad148b2762c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f83d2e3168149f3aa4fad148b2762c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33153 } } }
I20260810 22:43:30.324379 11746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3 [sys.catalog]: This master's current role is: LEADER
W20260810 22:43:30.326180 11763 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:30.326254 11763 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:30.326334 11754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:30.327028 11754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:30.332439 11754 catalog_manager.cc:1347] Generated new cluster ID: 7364dd82200e4760af20d12e01cb5538
I20260810 22:43:30.332506 11754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:30.351300 11754 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:30.351683 11754 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:30.360809 11754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f83d2e3168149f3aa4fad148b2762c3: Generated new TSK 0
I20260810 22:43:30.361622 11754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:30.375741 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11656
2026-08-10T22:43:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:30 dist-test-slave-1x4g krb5kdc[11770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:30 dist-test-slave-1x4g krb5kdc[11770](info): set up 2 sockets
Aug 10 22:43:30 dist-test-slave-1x4g krb5kdc[11770](info): commencing operation
krb5kdc: starting...
W20260810 22:43:31.459045 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:31 dist-test-slave-1x4g krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:31.583681 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41667
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:41667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:31.694218 11786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:31.694526 11786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:31.694605 11786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:31.698560 11786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:31.698652 11786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:31.698693 11786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:31.698750 11786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:31.698791 11786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:31.703894 11786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:41667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:41667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:31.704898 11786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:31.705896 11786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:31.714473 11796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:31.714587 11786 server_base.cc:1034] running on GCE node
W20260810 22:43:31.714514 11794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:31.714479 11791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:31.715250 11786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:31.716390 11786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:31.717595 11786 hybrid_clock.cc:648] HybridClock initialized: now 1786401811717555 us; error 52 us; skew 500 ppm
Aug 10 22:43:31 dist-test-slave-1x4g krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401811, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:31.720469 11786 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:31.721491 11786 webserver.cc:460] Webserver started at http://127.19.202.126:38029/ using document root <none> and password file <none>
I20260810 22:43:31.722095 11786 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:31.722199 11786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:31.722455 11786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:31.724545 11786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "6dfe17c7f81f40bbb74df7048ce722ae"
format_stamp: "Formatted at 2026-08-10 22:43:31 on dist-test-slave-1x4g"
server_key: "edc00e84a1f1c5ca42f724496b3caea1"
server_key_iv: "fda946827f8625567665a305a69cf810"
server_key_version: "encryptionkey@0"
I20260810 22:43:31.725204 11786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "6dfe17c7f81f40bbb74df7048ce722ae"
format_stamp: "Formatted at 2026-08-10 22:43:31 on dist-test-slave-1x4g"
server_key: "edc00e84a1f1c5ca42f724496b3caea1"
server_key_iv: "fda946827f8625567665a305a69cf810"
server_key_version: "encryptionkey@0"
I20260810 22:43:31.728935 11786 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:31.731269 11803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:31.732460 11786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:31.732610 11786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "6dfe17c7f81f40bbb74df7048ce722ae"
format_stamp: "Formatted at 2026-08-10 22:43:31 on dist-test-slave-1x4g"
server_key: "edc00e84a1f1c5ca42f724496b3caea1"
server_key_iv: "fda946827f8625567665a305a69cf810"
server_key_version: "encryptionkey@0"
I20260810 22:43:31.732728 11786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:31.756136 11786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:31.759085 11786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:31.759322 11786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:31.766824 11786 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:41667
I20260810 22:43:31.766834 11862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:41667 every 8 connection(s)
I20260810 22:43:31.767783 11786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:31.770699 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11786
I20260810 22:43:31.770766 11863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:31.770848 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:31.771173 20265 external_mini_cluster.cc:1428] Setting key c7ea24ae8bdbefe068dd0e634116848b
I20260810 22:43:31.777561 11863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae: Bootstrap starting.
I20260810 22:43:31.780284 11863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae: Neither blocks nor log segments found. Creating new log.
Aug 10 22:43:31 dist-test-slave-1x4g krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:31.781112 11863 log.cc:824] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:31.783495 11863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae: No bootstrap required, opened a new log
I20260810 22:43:31.786841 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:31.772539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58881 (local address 127.19.202.126:41667)
0810 22:43:31.773000 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:31.773018 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:31.773055 (+ 37us) server_negotiation.cc:407] Connection header received
0810 22:43:31.773967 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:31.773986 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:31.774311 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:31.774597 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:31.775506 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:31.776477 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:31.777700 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:31.778211 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:31.778346 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:31.781304 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:31.781310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:31.781325 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:31.781354 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:31.783422 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:31.784120 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:31.784126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:31.784131 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:31.784271 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:31.784602 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:31.784605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:31.784607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:31.785121 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:31.785327 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:31.785680 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 22:43:31.785937 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":166,"threads_started":1}
I20260810 22:43:31.788015 11863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41667 } }
I20260810 22:43:31.788225 11863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:31.788277 11863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dfe17c7f81f40bbb74df7048ce722ae, State: Initialized, Role: FOLLOWER
I20260810 22:43:31.788763 11863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41667 } }
I20260810 22:43:31.788969 11863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:31.789040 11863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:31.789140 11863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:31.789920 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:31.790097 11863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41667 } }
I20260810 22:43:31.790442 11863 leader_election.cc:302] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dfe17c7f81f40bbb74df7048ce722ae; no voters:
I20260810 22:43:31.790721 11863 leader_election.cc:288] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:31.790848 11868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:31.791169 11868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [term 1 LEADER]: Becoming Leader. State: Replica: 6dfe17c7f81f40bbb74df7048ce722ae, State: Running, Role: LEADER
I20260810 22:43:31.791532 11868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41667 } }
I20260810 22:43:31.791978 11863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:31.792832 11868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dfe17c7f81f40bbb74df7048ce722ae. Latest consensus state: current_term: 1 leader_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41667 } } }
I20260810 22:43:31.793041 11868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:31.793257 11869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfe17c7f81f40bbb74df7048ce722ae" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 41667 } } }
I20260810 22:43:31.793375 11869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:31.793656 11884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:31.797255 11884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:31.801400 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:31.792263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57909 (local address 127.19.202.126:41667)
0810 22:43:31.792383 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:31.792396 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:31.792753 (+ 357us) server_negotiation.cc:407] Connection header received
0810 22:43:31.793013 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:31.793018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:31.793066 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:31.793125 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:31.794136 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:31.794869 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:31.795975 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:31.796317 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:31.796428 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:31.798166 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:31.798170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:31.798172 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:31.798200 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:31.799466 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:31.799976 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:31.799979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:31.799981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:31.800104 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:31.800426 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:31.800429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:31.800430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:31.800593 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:31.800740 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:31.801123 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 22:43:31.801215 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 22:43:31.804279 11884 catalog_manager.cc:1347] Generated new cluster ID: 36a3187cefcc4034a4b609c3a8ed3d13
I20260810 22:43:31.804354 11884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:31.849582 11884 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:31.850684 11884 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:31.855779 11884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dfe17c7f81f40bbb74df7048ce722ae: Generated new TSK 0
I20260810 22:43:31.856483 11884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:31.865216 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11786
W20260810 22:43:31.874460 11768 connection.cc:535] client connection to 127.19.202.126:41667 recv error: Network error: failed to read from TLS socket (remote: 127.19.202.126:41667): Connection reset by peer (error 104)
2026-08-10T22:43:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:31 dist-test-slave-1x4g krb5kdc[11899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:31 dist-test-slave-1x4g krb5kdc[11899](info): set up 2 sockets
Aug 10 22:43:31 dist-test-slave-1x4g krb5kdc[11899](info): commencing operation
krb5kdc: starting...
W20260810 22:43:32.958976 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:33 dist-test-slave-1x4g krb5kdc[11899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:33.079896 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37931
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:37931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:33.188791 11918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:33.189070 11918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:33.189136 11918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:33.192914 11918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:33.193004 11918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:33.193041 11918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:33.193094 11918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:33.193132 11918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:33.198041 11918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:37931
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:37931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:33.199019 11918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:33.199962 11918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:33.208271 11924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:33.208271 11925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:33.208287 11927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:33.208338 11918 server_base.cc:1034] running on GCE node
I20260810 22:43:33.209131 11918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:33.210191 11918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:33.211373 11918 hybrid_clock.cc:648] HybridClock initialized: now 1786401813211361 us; error 34 us; skew 500 ppm
Aug 10 22:43:33 dist-test-slave-1x4g krb5kdc[11899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401813, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:33.214111 11918 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:33.214996 11918 webserver.cc:460] Webserver started at http://127.19.202.126:35925/ using document root <none> and password file <none>
I20260810 22:43:33.215555 11918 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:33.215642 11918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:33.215870 11918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:33.217945 11918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "ced58b3d6a304c77b1332e96ce32cc19"
format_stamp: "Formatted at 2026-08-10 22:43:33 on dist-test-slave-1x4g"
server_key: "324ac6e6b9653a75d878877fef3fc4d1"
server_key_iv: "f4970f86148d24a493c19c01d2f01f71"
server_key_version: "encryptionkey@0"
I20260810 22:43:33.218519 11918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "ced58b3d6a304c77b1332e96ce32cc19"
format_stamp: "Formatted at 2026-08-10 22:43:33 on dist-test-slave-1x4g"
server_key: "324ac6e6b9653a75d878877fef3fc4d1"
server_key_iv: "f4970f86148d24a493c19c01d2f01f71"
server_key_version: "encryptionkey@0"
I20260810 22:43:33.222160 11918 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:33.224481 11936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:33.225680 11918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:33.225863 11918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "ced58b3d6a304c77b1332e96ce32cc19"
format_stamp: "Formatted at 2026-08-10 22:43:33 on dist-test-slave-1x4g"
server_key: "324ac6e6b9653a75d878877fef3fc4d1"
server_key_iv: "f4970f86148d24a493c19c01d2f01f71"
server_key_version: "encryptionkey@0"
I20260810 22:43:33.226008 11918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:33.240464 11918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:33.243233 11918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:33.243420 11918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:33.250519 11918 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:37931
I20260810 22:43:33.250530 11996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:37931 every 8 connection(s)
I20260810 22:43:33.251444 11918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:33.254199 11999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:33.256512 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 11918
I20260810 22:43:33.256690 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:33.256996 20265 external_mini_cluster.cc:1428] Setting key 1860eccc934f105ff252ad55c515eefb
I20260810 22:43:33.260468 11999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19: Bootstrap starting.
I20260810 22:43:33.263098 11999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:33.264005 11999 log.cc:824] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:33.265966 11999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19: No bootstrap required, opened a new log
I20260810 22:43:33.266903 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:33.258405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43233 (local address 127.19.202.126:37931)
0810 22:43:33.258815 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:33.258834 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:33.258877 (+ 43us) server_negotiation.cc:407] Connection header received
0810 22:43:33.259726 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:33.259750 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:33.260161 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:33.260520 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:33.260979 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:33.262466 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:33.263389 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:33.264087 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:33.264249 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:33.265273 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:33.265278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:33.265290 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:33.265662 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:33.265713 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:33.265736 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:33.265876 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:33.266070 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 22:43:33.266213 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":95,"threads_started":1}
I20260810 22:43:33.269991 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:33.270004 11999 raft_consensus.cc:348] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ced58b3d6a304c77b1332e96ce32cc19" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37931 } }
I20260810 22:43:33.270292 11999 raft_consensus.cc:374] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:33.270381 11999 raft_consensus.cc:729] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ced58b3d6a304c77b1332e96ce32cc19, State: Initialized, Role: FOLLOWER
I20260810 22:43:33.270834 11999 consensus_queue.cc:260] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ced58b3d6a304c77b1332e96ce32cc19" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37931 } }
I20260810 22:43:33.270989 11999 raft_consensus.cc:388] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:33.271061 11999 raft_consensus.cc:482] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:33.271157 11999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:33.272089 11999 raft_consensus.cc:504] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ced58b3d6a304c77b1332e96ce32cc19" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37931 } }
I20260810 22:43:33.272461 11999 leader_election.cc:302] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ced58b3d6a304c77b1332e96ce32cc19; no voters:
I20260810 22:43:33.272825 11999 leader_election.cc:288] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:33.272948 12009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:33.273255 12009 raft_consensus.cc:686] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [term 1 LEADER]: Becoming Leader. State: Replica: ced58b3d6a304c77b1332e96ce32cc19, State: Running, Role: LEADER
I20260810 22:43:33.273622 12009 consensus_queue.cc:237] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ced58b3d6a304c77b1332e96ce32cc19" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37931 } }
I20260810 22:43:33.274102 11999 sys_catalog.cc:564] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:33.274807 12009 sys_catalog.cc:455] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ced58b3d6a304c77b1332e96ce32cc19. Latest consensus state: current_term: 1 leader_uuid: "ced58b3d6a304c77b1332e96ce32cc19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ced58b3d6a304c77b1332e96ce32cc19" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37931 } } }
I20260810 22:43:33.275036 12009 sys_catalog.cc:458] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:33.275233 12010 sys_catalog.cc:455] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ced58b3d6a304c77b1332e96ce32cc19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ced58b3d6a304c77b1332e96ce32cc19" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 37931 } } }
I20260810 22:43:33.275496 12010 sys_catalog.cc:458] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:33.275518 12022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:33.277428 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:33.272363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34521 (local address 127.19.202.126:37931)
0810 22:43:33.272503 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:33.272508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:33.272712 (+ 204us) server_negotiation.cc:407] Connection header received
0810 22:43:33.272866 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:33.272871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:33.272931 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:33.273030 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:33.273315 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:33.274059 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:33.275290 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:33.275657 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:33.275748 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:33.276892 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:33.276903 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:33.276905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:33.276918 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:33.276947 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:33.276958 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:33.277054 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:33.277254 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 22:43:33.277291 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:43:33.279125 12022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:33.285029 12022 catalog_manager.cc:1347] Generated new cluster ID: 9a4291707aa44d23891684c613959b8a
I20260810 22:43:33.285094 12022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:33.308653 12022 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:33.309628 12022 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:33.315675 12022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ced58b3d6a304c77b1332e96ce32cc19: Generated new TSK 0
I20260810 22:43:33.316318 12022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:33.324885 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 11918
W20260810 22:43:33.332613 11896 connection.cc:535] client connection to 127.19.202.126:37931 recv error: Network error: failed to read from TLS socket (remote: 127.19.202.126:37931): Connection reset by peer (error 104)
2026-08-10T22:43:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:33 dist-test-slave-1x4g krb5kdc[12038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:33 dist-test-slave-1x4g krb5kdc[12038](info): set up 2 sockets
Aug 10 22:43:33 dist-test-slave-1x4g krb5kdc[12038](info): commencing operation
krb5kdc: starting...
W20260810 22:43:34.421190 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:34 dist-test-slave-1x4g krb5kdc[12038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:34.547976 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:44971
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:44971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:34.660251 12052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:34.660545 12052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:34.660626 12052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:34.664547 12052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:34.664646 12052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:34.664685 12052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:34.664721 12052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:34.664757 12052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:34.669812 12052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:44971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:34.670851 12052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:34.671936 12052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:34.680328 12057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:34.680349 12058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:34.680495 12052 server_base.cc:1034] running on GCE node
W20260810 22:43:34.680677 12060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:34.681277 12052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:34.682353 12052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:34.683563 12052 hybrid_clock.cc:648] HybridClock initialized: now 1786401814683544 us; error 30 us; skew 500 ppm
Aug 10 22:43:34 dist-test-slave-1x4g krb5kdc[12038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401814, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:34.690984 12052 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:34.691874 12052 webserver.cc:460] Webserver started at http://127.19.202.126:37685/ using document root <none> and password file <none>
I20260810 22:43:34.692456 12052 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:34.692546 12052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:34.692831 12052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:34.695014 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "91a11f0838b642caba9c5bfa66db3c80"
format_stamp: "Formatted at 2026-08-10 22:43:34 on dist-test-slave-1x4g"
server_key: "9e98dee76ed860d9c8cf8ff2dfe1bc3f"
server_key_iv: "0d9e7e6de008792d44cad77790164618"
server_key_version: "encryptionkey@0"
I20260810 22:43:34.695611 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "91a11f0838b642caba9c5bfa66db3c80"
format_stamp: "Formatted at 2026-08-10 22:43:34 on dist-test-slave-1x4g"
server_key: "9e98dee76ed860d9c8cf8ff2dfe1bc3f"
server_key_iv: "0d9e7e6de008792d44cad77790164618"
server_key_version: "encryptionkey@0"
I20260810 22:43:34.699415 12052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 22:43:34.701826 12067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:34.703030 12052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:34.703161 12052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "91a11f0838b642caba9c5bfa66db3c80"
format_stamp: "Formatted at 2026-08-10 22:43:34 on dist-test-slave-1x4g"
server_key: "9e98dee76ed860d9c8cf8ff2dfe1bc3f"
server_key_iv: "0d9e7e6de008792d44cad77790164618"
server_key_version: "encryptionkey@0"
I20260810 22:43:34.703267 12052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:34.707836 12052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:34.710716 12052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:34.710922 12052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:34.718076 12052 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:44971
I20260810 22:43:34.718104 12126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:44971 every 8 connection(s)
I20260810 22:43:34.719009 12052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:34.721711 12127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:34.724014 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 12052
I20260810 22:43:34.724175 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:34.724505 20265 external_mini_cluster.cc:1428] Setting key b4b2f4cd44f24af3e2e5a5d8f5cb9615
I20260810 22:43:34.727950 12127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80: Bootstrap starting.
I20260810 22:43:34.729768 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:34.725899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56687 (local address 127.19.202.126:44971)
0810 22:43:34.726365 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:34.726380 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:34.726416 (+ 36us) server_negotiation.cc:407] Connection header received
0810 22:43:34.727057 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:34.727073 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:34.727379 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:34.727774 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:34.728572 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:34.728577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:34.728589 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:34.728599 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:34.728651 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:34.728671 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:34.728826 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:34.728938 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 22:43:34.729091 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":140,"threads_started":1}
I20260810 22:43:34.730404 12127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:34.731287 12127 log.cc:824] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:34.732664 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:34.733457 12127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80: No bootstrap required, opened a new log
I20260810 22:43:34.736284 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:34.734718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48907 (local address 127.19.202.126:44971)
0810 22:43:34.734829 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:34.734835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:34.734973 (+ 138us) server_negotiation.cc:407] Connection header received
0810 22:43:34.735105 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:34.735107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:34.735134 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:34.735197 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:34.735874 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:34.735877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:34.735879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:34.735889 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:34.735912 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:34.735922 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:34.735997 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:34.736110 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 22:43:34.736141 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:43:34.737924 12127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a11f0838b642caba9c5bfa66db3c80" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 44971 } }
I20260810 22:43:34.738142 12127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:34.738217 12127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a11f0838b642caba9c5bfa66db3c80, State: Initialized, Role: FOLLOWER
I20260810 22:43:34.738619 12127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a11f0838b642caba9c5bfa66db3c80" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 44971 } }
I20260810 22:43:34.738755 12127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:34.738821 12127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:34.738935 12127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:34.739812 12127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a11f0838b642caba9c5bfa66db3c80" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 44971 } }
I20260810 22:43:34.740147 12127 leader_election.cc:302] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91a11f0838b642caba9c5bfa66db3c80; no voters:
I20260810 22:43:34.740442 12127 leader_election.cc:288] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:34.740542 12137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:34.740814 12137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [term 1 LEADER]: Becoming Leader. State: Replica: 91a11f0838b642caba9c5bfa66db3c80, State: Running, Role: LEADER
I20260810 22:43:34.741129 12137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a11f0838b642caba9c5bfa66db3c80" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 44971 } }
I20260810 22:43:34.741725 12127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:34.742873 12140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a11f0838b642caba9c5bfa66db3c80. Latest consensus state: current_term: 1 leader_uuid: "91a11f0838b642caba9c5bfa66db3c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a11f0838b642caba9c5bfa66db3c80" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 44971 } } }
I20260810 22:43:34.742990 12140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:34.742842 12139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91a11f0838b642caba9c5bfa66db3c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a11f0838b642caba9c5bfa66db3c80" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 44971 } } }
I20260810 22:43:34.743244 12139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:34.743546 12147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:34.746690 12147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:34.751844 12147 catalog_manager.cc:1347] Generated new cluster ID: 5b58872f6f234770ac0a85268e81abb7
I20260810 22:43:34.751912 12147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:34.764487 12147 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:34.764839 12147 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:34.770859 12147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91a11f0838b642caba9c5bfa66db3c80: Generated new TSK 0
I20260810 22:43:34.771487 12147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:34.779507 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 12052
2026-08-10T22:43:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:34 dist-test-slave-1x4g krb5kdc[12160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:34 dist-test-slave-1x4g krb5kdc[12160](info): set up 2 sockets
Aug 10 22:43:34 dist-test-slave-1x4g krb5kdc[12160](info): commencing operation
krb5kdc: starting...
W20260810 22:43:35.872079 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:35 dist-test-slave-1x4g krb5kdc[12160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:35.994234 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:40067
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:40067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:36.105201 12177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:36.105489 12177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:36.105530 12177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:36.109338 12177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:36.109391 12177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:36.109411 12177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:36.109426 12177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:36.109440 12177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:36.114460 12177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:40067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:40067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:36.115476 12177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:36.116478 12177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:36.124974 12184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:36.125012 12187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:36.125155 12177 server_base.cc:1034] running on GCE node
W20260810 22:43:36.124975 12185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:36.125818 12177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:36.126917 12177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:36.128095 12177 hybrid_clock.cc:648] HybridClock initialized: now 1786401816128062 us; error 46 us; skew 500 ppm
Aug 10 22:43:36 dist-test-slave-1x4g krb5kdc[12160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401816, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:36.131017 12177 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:36.131878 12177 webserver.cc:460] Webserver started at http://127.19.202.126:40961/ using document root <none> and password file <none>
I20260810 22:43:36.132467 12177 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:36.132557 12177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:36.132813 12177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:36.134852 12177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "671d6f76fbc447f894b81e08e3830416"
format_stamp: "Formatted at 2026-08-10 22:43:36 on dist-test-slave-1x4g"
server_key: "65e87ab730bf93558a31766c94714ed9"
server_key_iv: "99be167accc09d49fe37f72e7df7300f"
server_key_version: "encryptionkey@0"
I20260810 22:43:36.135422 12177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "671d6f76fbc447f894b81e08e3830416"
format_stamp: "Formatted at 2026-08-10 22:43:36 on dist-test-slave-1x4g"
server_key: "65e87ab730bf93558a31766c94714ed9"
server_key_iv: "99be167accc09d49fe37f72e7df7300f"
server_key_version: "encryptionkey@0"
I20260810 22:43:36.139099 12177 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 22:43:36.141450 12195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:36.142632 12177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:36.142760 12177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "671d6f76fbc447f894b81e08e3830416"
format_stamp: "Formatted at 2026-08-10 22:43:36 on dist-test-slave-1x4g"
server_key: "65e87ab730bf93558a31766c94714ed9"
server_key_iv: "99be167accc09d49fe37f72e7df7300f"
server_key_version: "encryptionkey@0"
I20260810 22:43:36.142864 12177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:36.154691 12177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:36.157464 12177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:36.157655 12177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:36.164389 12177 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:40067
I20260810 22:43:36.164409 12253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:40067 every 8 connection(s)
I20260810 22:43:36.165300 12177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:36.168031 12254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:36.170107 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 12177
I20260810 22:43:36.170249 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:36.170562 20265 external_mini_cluster.cc:1428] Setting key 4fc2509d1a95b97fa01b5c46be5b64f3
I20260810 22:43:36.174875 12254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416: Bootstrap starting.
I20260810 22:43:36.177457 12254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:36.178391 12254 log.cc:824] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416: Log is configured to *not* fsync() on all Append() calls
Aug 10 22:43:36 dist-test-slave-1x4g krb5kdc[12160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:36.180403 12254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416: No bootstrap required, opened a new log
I20260810 22:43:36.184005 12254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d6f76fbc447f894b81e08e3830416" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40067 } }
I20260810 22:43:36.184238 12254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:36.184309 12254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d6f76fbc447f894b81e08e3830416, State: Initialized, Role: FOLLOWER
I20260810 22:43:36.184726 12254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d6f76fbc447f894b81e08e3830416" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40067 } }
I20260810 22:43:36.184826 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:36.171844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48055 (local address 127.19.202.126:40067)
0810 22:43:36.172239 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:36.172254 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:36.172289 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:43:36.172979 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:36.172996 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:36.173354 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:36.173636 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:36.174619 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:36.175639 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:36.176890 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:36.177333 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:36.177457 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:36.180266 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:36.180271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:36.180283 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:36.180309 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:36.181994 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:36.182418 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:36.182424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:36.182430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:36.182540 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:36.182812 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:36.182816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:36.182822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:36.183201 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:36.183410 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:36.183696 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 22:43:36.183968 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":120,"threads_started":1}
I20260810 22:43:36.184933 12254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:36.185034 12254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:36.185169 12254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:36.186146 12254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d6f76fbc447f894b81e08e3830416" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40067 } }
I20260810 22:43:36.186463 12254 leader_election.cc:302] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 671d6f76fbc447f894b81e08e3830416; no voters:
I20260810 22:43:36.186820 12254 leader_election.cc:288] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:36.186906 12260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:36.187114 12260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [term 1 LEADER]: Becoming Leader. State: Replica: 671d6f76fbc447f894b81e08e3830416, State: Running, Role: LEADER
I20260810 22:43:36.187461 12260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d6f76fbc447f894b81e08e3830416" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40067 } }
I20260810 22:43:36.187876 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:36.188149 12254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:36.189371 12262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671d6f76fbc447f894b81e08e3830416. Latest consensus state: current_term: 1 leader_uuid: "671d6f76fbc447f894b81e08e3830416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d6f76fbc447f894b81e08e3830416" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40067 } } }
I20260810 22:43:36.189507 12262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:36.190407 12261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "671d6f76fbc447f894b81e08e3830416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d6f76fbc447f894b81e08e3830416" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 40067 } } }
I20260810 22:43:36.190552 12261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:36.190908 12277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:36.193758 12277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:36.199327 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:36.190302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49207 (local address 127.19.202.126:40067)
0810 22:43:36.190430 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:36.190434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:36.190823 (+ 389us) server_negotiation.cc:407] Connection header received
0810 22:43:36.191014 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:36.191019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:36.191068 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:36.191109 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:36.191799 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:36.192568 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:36.193575 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:36.193843 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:36.193949 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:36.195259 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:36.195263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:36.195265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:36.195290 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:36.196997 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:36.197894 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:36.197898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:36.197900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:36.197989 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:36.198331 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:36.198334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:36.198336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:36.198567 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:36.198671 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:36.199038 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 22:43:36.199154 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 22:43:36.199734 12277 catalog_manager.cc:1347] Generated new cluster ID: 3cd59364d31d4ac7909c29f94fc1cab8
I20260810 22:43:36.199833 12277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:36.211194 12277 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:36.212304 12277 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:36.223009 12277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 671d6f76fbc447f894b81e08e3830416: Generated new TSK 0
I20260810 22:43:36.223839 12277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:36.232744 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 12177
2026-08-10T22:43:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:36 dist-test-slave-1x4g krb5kdc[12292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:36 dist-test-slave-1x4g krb5kdc[12292](info): set up 2 sockets
Aug 10 22:43:36 dist-test-slave-1x4g krb5kdc[12292](info): commencing operation
krb5kdc: starting...
W20260810 22:43:37.334765 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:37 dist-test-slave-1x4g krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:37.463716 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:42929
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:42929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:37.575591 12307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:37.575893 12307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:37.575969 12307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:37.580025 12307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:37.580119 12307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:37.580158 12307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:37.580214 12307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:37.580247 12307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:37.585440 12307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:42929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:42929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:37.586475 12307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:37.587504 12307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:37.596251 12314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:37.596256 12313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:37.596251 12319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:37.596463 12307 server_base.cc:1034] running on GCE node
I20260810 22:43:37.597168 12307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:37.598228 12307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:37.599403 12307 hybrid_clock.cc:648] HybridClock initialized: now 1786401817599389 us; error 25 us; skew 500 ppm
Aug 10 22:43:37 dist-test-slave-1x4g krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401817, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:37.602247 12307 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:37.603125 12307 webserver.cc:460] Webserver started at http://127.19.202.126:40615/ using document root <none> and password file <none>
I20260810 22:43:37.603681 12307 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:37.603765 12307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:37.603991 12307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:37.606010 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "9606c162edad45ec9267657217d9a4b1"
format_stamp: "Formatted at 2026-08-10 22:43:37 on dist-test-slave-1x4g"
server_key: "c5a43fae08e4c889b8b35ce55aceb24d"
server_key_iv: "b208a5c7fd80d7797d6dd61c388c7bf2"
server_key_version: "encryptionkey@0"
I20260810 22:43:37.606566 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "9606c162edad45ec9267657217d9a4b1"
format_stamp: "Formatted at 2026-08-10 22:43:37 on dist-test-slave-1x4g"
server_key: "c5a43fae08e4c889b8b35ce55aceb24d"
server_key_iv: "b208a5c7fd80d7797d6dd61c388c7bf2"
server_key_version: "encryptionkey@0"
I20260810 22:43:37.610176 12307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260810 22:43:37.612478 12327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:37.613695 12307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 22:43:37.613859 12307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "9606c162edad45ec9267657217d9a4b1"
format_stamp: "Formatted at 2026-08-10 22:43:37 on dist-test-slave-1x4g"
server_key: "c5a43fae08e4c889b8b35ce55aceb24d"
server_key_iv: "b208a5c7fd80d7797d6dd61c388c7bf2"
server_key_version: "encryptionkey@0"
I20260810 22:43:37.613977 12307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:37.634100 12307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:37.637063 12307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:37.637255 12307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:37.644572 12307 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:42929
I20260810 22:43:37.644601 12383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:42929 every 8 connection(s)
I20260810 22:43:37.645587 12307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:37.648298 12384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:37.649933 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 12307
I20260810 22:43:37.650082 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:37.650384 20265 external_mini_cluster.cc:1428] Setting key ef8e158422cee2a3929976cf70e49867
I20260810 22:43:37.654495 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1: Bootstrap starting.
I20260810 22:43:37.656631 12384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:37.657445 12384 log.cc:824] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:37.659096 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:37.651810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56267 (local address 127.19.202.126:42929)
0810 22:43:37.652273 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:37.652289 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:37.652324 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:43:37.653024 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:37.653042 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:37.653375 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:37.653701 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:37.654097 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:37.655084 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:37.655954 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:37.656408 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:37.656640 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:37.657488 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:37.657493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:37.657505 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:37.657911 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:37.657968 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:37.657991 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:37.658096 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:37.658225 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 22:43:37.658429 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":151,"threads_started":1}
I20260810 22:43:37.659348 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1: No bootstrap required, opened a new log
I20260810 22:43:37.661794 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:37.663348 12384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9606c162edad45ec9267657217d9a4b1" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42929 } }
I20260810 22:43:37.663661 12384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:37.663743 12384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9606c162edad45ec9267657217d9a4b1, State: Initialized, Role: FOLLOWER
I20260810 22:43:37.664204 12384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9606c162edad45ec9267657217d9a4b1" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42929 } }
I20260810 22:43:37.664372 12384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:37.664449 12384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:37.664572 12384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:37.665671 12384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9606c162edad45ec9267657217d9a4b1" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42929 } }
I20260810 22:43:37.666002 12384 leader_election.cc:302] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9606c162edad45ec9267657217d9a4b1; no voters:
I20260810 22:43:37.666311 12384 leader_election.cc:288] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:37.666422 12395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:37.666638 12395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [term 1 LEADER]: Becoming Leader. State: Replica: 9606c162edad45ec9267657217d9a4b1, State: Running, Role: LEADER
I20260810 22:43:37.666988 12395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9606c162edad45ec9267657217d9a4b1" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42929 } }
I20260810 22:43:37.667625 12384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:37.668234 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:37.663850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47923 (local address 127.19.202.126:42929)
0810 22:43:37.663975 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:37.663978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:37.664178 (+ 200us) server_negotiation.cc:407] Connection header received
0810 22:43:37.664315 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:37.664317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:37.664375 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:37.664452 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:37.664716 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:37.665484 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:37.666381 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:37.666715 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:37.666813 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:37.667799 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:37.667802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:37.667805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:37.667821 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:37.667846 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:37.667855 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:37.667932 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:37.668027 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 22:43:37.668087 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 22:43:37.668818 12396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9606c162edad45ec9267657217d9a4b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9606c162edad45ec9267657217d9a4b1" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42929 } } }
I20260810 22:43:37.668943 12397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9606c162edad45ec9267657217d9a4b1. Latest consensus state: current_term: 1 leader_uuid: "9606c162edad45ec9267657217d9a4b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9606c162edad45ec9267657217d9a4b1" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 42929 } } }
I20260810 22:43:37.669040 12396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:37.669093 12397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:37.669533 12407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:37.672351 12407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:37.677196 12407 catalog_manager.cc:1347] Generated new cluster ID: cf22371a6adf4633b041d25772089de1
I20260810 22:43:37.677259 12407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:37.689767 12407 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:37.690623 12407 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:37.703533 12407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9606c162edad45ec9267657217d9a4b1: Generated new TSK 0
I20260810 22:43:37.704324 12407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:37.712337 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 12307
2026-08-10T22:43:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:37 dist-test-slave-1x4g krb5kdc[12420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:37 dist-test-slave-1x4g krb5kdc[12420](info): set up 2 sockets
Aug 10 22:43:37 dist-test-slave-1x4g krb5kdc[12420](info): commencing operation
krb5kdc: starting...
W20260810 22:43:38.804610 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:38 dist-test-slave-1x4g krb5kdc[12420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:38.929788 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38759
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:35605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:38759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:39.038275 12437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:39.038515 12437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:39.038556 12437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:39.042410 12437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:39.042461 12437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:39.042481 12437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:39.042498 12437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:39.042552 12437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:39.047633 12437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:35605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:38759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:38759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:39.048629 12437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:39.049681 12437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:39.058207 12442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:39.058300 12437 server_base.cc:1034] running on GCE node
W20260810 22:43:39.058234 12445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:39.058207 12443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:39.059108 12437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:39.060170 12437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:39.061363 12437 hybrid_clock.cc:648] HybridClock initialized: now 1786401819061338 us; error 35 us; skew 500 ppm
Aug 10 22:43:39 dist-test-slave-1x4g krb5kdc[12420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401819, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:39.064214 12437 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:39.065161 12437 webserver.cc:460] Webserver started at http://127.19.202.126:41847/ using document root <none> and password file <none>
I20260810 22:43:39.065737 12437 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:39.065826 12437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:39.066061 12437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:39.068261 12437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "dea4eae4c94141daa84922e0e9471aa6"
format_stamp: "Formatted at 2026-08-10 22:43:39 on dist-test-slave-1x4g"
server_key: "c4247f88ac658779b6cbeb2e297e24e9"
server_key_iv: "f68d44604948821b803a6b0eebe7e250"
server_key_version: "encryptionkey@0"
I20260810 22:43:39.068928 12437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "dea4eae4c94141daa84922e0e9471aa6"
format_stamp: "Formatted at 2026-08-10 22:43:39 on dist-test-slave-1x4g"
server_key: "c4247f88ac658779b6cbeb2e297e24e9"
server_key_iv: "f68d44604948821b803a6b0eebe7e250"
server_key_version: "encryptionkey@0"
I20260810 22:43:39.072491 12437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 22:43:39.074922 12453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:39.076092 12437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:39.076231 12437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "dea4eae4c94141daa84922e0e9471aa6"
format_stamp: "Formatted at 2026-08-10 22:43:39 on dist-test-slave-1x4g"
server_key: "c4247f88ac658779b6cbeb2e297e24e9"
server_key_iv: "f68d44604948821b803a6b0eebe7e250"
server_key_version: "encryptionkey@0"
I20260810 22:43:39.076333 12437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:39.080955 12437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:39.084048 12437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:39.084232 12437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:39.091259 12437 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:38759
I20260810 22:43:39.091287 12509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:38759 every 8 connection(s)
I20260810 22:43:39.092186 12437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:39.094910 12510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:39.095252 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 12437
I20260810 22:43:39.095412 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:39.095711 20265 external_mini_cluster.cc:1428] Setting key ee0e55a2864fad539ce1c10403540ec3
I20260810 22:43:39.100922 12513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:39.097128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32791 (local address 127.19.202.126:38759)
0810 22:43:39.097585 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:39.097600 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:39.097634 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:43:39.098323 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:39.098340 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:39.098615 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:39.098942 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:39.099735 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:39.099741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:39.099760 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:39.099771 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:39.099825 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:39.099844 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:39.099987 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:39.100055 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 22:43:39.100202 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":165,"threads_started":1}
I20260810 22:43:39.102028 12510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6: Bootstrap starting.
I20260810 22:43:39.103749 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:39.104558 12510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:39.105456 12510 log.cc:824] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:39.107430 12513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:39.105828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49551 (local address 127.19.202.126:38759)
0810 22:43:39.105944 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:39.105948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:39.106162 (+ 214us) server_negotiation.cc:407] Connection header received
0810 22:43:39.106271 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:39.106274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:39.106304 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:39.106364 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:39.107014 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:39.107017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:39.107019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:43:39.107029 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:39.107068 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:43:39.107078 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:39.107143 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:39.107260 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 22:43:39.107290 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 22:43:39.107801 12510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6: No bootstrap required, opened a new log
I20260810 22:43:39.111297 12510 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea4eae4c94141daa84922e0e9471aa6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38759 } }
I20260810 22:43:39.111553 12510 raft_consensus.cc:374] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:43:39.111634 12510 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea4eae4c94141daa84922e0e9471aa6, State: Initialized, Role: FOLLOWER
I20260810 22:43:39.112092 12510 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea4eae4c94141daa84922e0e9471aa6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38759 } }
I20260810 22:43:39.112272 12510 raft_consensus.cc:388] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:43:39.112362 12510 raft_consensus.cc:482] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:43:39.112469 12510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:39.113477 12510 raft_consensus.cc:504] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea4eae4c94141daa84922e0e9471aa6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38759 } }
I20260810 22:43:39.113869 12510 leader_election.cc:302] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dea4eae4c94141daa84922e0e9471aa6; no voters:
I20260810 22:43:39.114192 12510 leader_election.cc:288] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:43:39.114357 12521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:39.114614 12521 raft_consensus.cc:686] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [term 1 LEADER]: Becoming Leader. State: Replica: dea4eae4c94141daa84922e0e9471aa6, State: Running, Role: LEADER
I20260810 22:43:39.114933 12521 consensus_queue.cc:237] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea4eae4c94141daa84922e0e9471aa6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38759 } }
I20260810 22:43:39.115454 12510 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:39.116704 12522 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dea4eae4c94141daa84922e0e9471aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea4eae4c94141daa84922e0e9471aa6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38759 } } }
I20260810 22:43:39.116704 12524 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea4eae4c94141daa84922e0e9471aa6. Latest consensus state: current_term: 1 leader_uuid: "dea4eae4c94141daa84922e0e9471aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea4eae4c94141daa84922e0e9471aa6" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 38759 } } }
I20260810 22:43:39.116883 12522 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:39.116883 12524 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:39.117293 12532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:39.120340 12532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:39.125679 12532 catalog_manager.cc:1347] Generated new cluster ID: 6655000d3eea46a0b5ffc96e59762784
I20260810 22:43:39.125757 12532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:39.177273 12532 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:39.177611 12532 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:39.187274 12532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dea4eae4c94141daa84922e0e9471aa6: Generated new TSK 0
I20260810 22:43:39.187948 12532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:43:39.200628 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 12437
2026-08-10T22:43:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1487 ms)
[----------] 18 tests from AuthTokenIssuingTest (26282 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:39 dist-test-slave-1x4g krb5kdc[12547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:39 dist-test-slave-1x4g krb5kdc[12547](info): set up 2 sockets
Aug 10 22:43:39 dist-test-slave-1x4g krb5kdc[12547](info): commencing operation
krb5kdc: starting...
W20260810 22:43:40.280097 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:40 dist-test-slave-1x4g krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:40.403239 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:46645
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:46645,127.19.202.125:42461,127.19.202.124:33223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:40.516178 12564 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 22:43:40.516479 12564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:40.516544 12564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:40.516680 12564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:40.520398 12564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:40.520473 12564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:40.520509 12564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:40.520541 12564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:40.520572 12564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:40.525579 12564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:46645,127.19.202.125:42461,127.19.202.124:33223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:46645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:40.526518 12564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:40.527530 12564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:43:40.536216 12564 server_base.cc:1034] running on GCE node
W20260810 22:43:40.536165 12572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:40.536156 12574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:40.536175 12570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:40.536998 12564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:40.538022 12564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:40.539208 12564 hybrid_clock.cc:648] HybridClock initialized: now 1786401820539190 us; error 32 us; skew 500 ppm
Aug 10 22:43:40 dist-test-slave-1x4g krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:40.546840 12564 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:40.547679 12564 webserver.cc:460] Webserver started at http://127.19.202.126:35421/ using document root <none> and password file <none>
I20260810 22:43:40.548195 12564 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:40.548302 12564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:40.548586 12564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:40.550590 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "d6406f11439642f0a16068d96a6ad160"
format_stamp: "Formatted at 2026-08-10 22:43:40 on dist-test-slave-1x4g"
server_key: "c8bcb1025535da5ecc030f0ec27c8f03"
server_key_iv: "69dc65a3911314134df7857e26f89a22"
server_key_version: "encryptionkey@0"
I20260810 22:43:40.551179 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "d6406f11439642f0a16068d96a6ad160"
format_stamp: "Formatted at 2026-08-10 22:43:40 on dist-test-slave-1x4g"
server_key: "c8bcb1025535da5ecc030f0ec27c8f03"
server_key_iv: "69dc65a3911314134df7857e26f89a22"
server_key_version: "encryptionkey@0"
I20260810 22:43:40.555217 12564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 22:43:40.557579 12581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:40.558730 12564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:40.558852 12564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "d6406f11439642f0a16068d96a6ad160"
format_stamp: "Formatted at 2026-08-10 22:43:40 on dist-test-slave-1x4g"
server_key: "c8bcb1025535da5ecc030f0ec27c8f03"
server_key_iv: "69dc65a3911314134df7857e26f89a22"
server_key_version: "encryptionkey@0"
I20260810 22:43:40.558982 12564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:40.587347 12564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:40.590190 12564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:40.590389 12564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:40.597343 12564 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:46645
I20260810 22:43:40.597622 12641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:46645 every 8 connection(s)
I20260810 22:43:40.598279 12564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:40.599318 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 12564
I20260810 22:43:40.599468 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:40.599759 20265 external_mini_cluster.cc:1428] Setting key e2969b287f1ff074e6292524e856a529
I20260810 22:43:40.601225 12642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:40.604830 12642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.202.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.125@KRBTEST.COM" created.
Aug 10 22:43:40 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:40.619486 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.607554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.19.202.126:46645)
0810 22:43:40.607860 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:40.607872 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:40.607932 (+ 60us) server_negotiation.cc:407] Connection header received
0810 22:43:40.608391 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:40.608396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:40.608633 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:40.608853 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:40.609631 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.611120 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.612098 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.612472 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.612588 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.614876 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:40.614882 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:40.614891 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:40.614910 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:40.616427 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.616983 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.616989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.616990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.617091 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.617408 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.617411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.617412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.617746 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:40.617908 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:40.618236 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 22:43:40.618506 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
I20260810 22:43:40.619484 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.606752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:46645 (local address 127.0.0.1:60782)
0810 22:43:40.607171 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.607217 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:40.608088 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:40.608892 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:40.608897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:40.609200 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:40.609479 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.609492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.611259 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.611262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.611997 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.612005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.612603 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.612606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.612698 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.613316 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:40.613332 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:40.614735 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:40.616544 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.616554 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.616563 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.616883 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.617198 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.617201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.617202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.617312 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.617871 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:40.617876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:40.618009 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 22:43:40.618156 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 22:43:40.618556 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":115,"thread_start_us":128,"threads_started":1}
I20260810 22:43:40.621810 12642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:40.622479 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.622200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57858)
0810 22:43:40.622323 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.622390 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 22:43:40.622714 12586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
I20260810 22:43:40.623521 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.623283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57870)
0810 22:43:40.623380 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.623455 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 22:43:40.623651 12642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:42461: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab.
Entry for principal kudu/127.19.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab.
I20260810 22:43:40.627969 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:42461
--webserver_interface=127.19.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab
--principal=kudu/127.19.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:46645,127.19.202.125:42461,127.19.202.124:33223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:43:40.672838 12642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } attempt: 1
I20260810 22:43:40.674921 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.674353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57878)
0810 22:43:40.674703 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.674810 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232}
I20260810 22:43:40.677260 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.676364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57894)
0810 22:43:40.676752 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.676855 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":110,"threads_started":1}
W20260810 22:43:40.677474 12642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:42461: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
W20260810 22:43:40.740073 12651 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 22:43:40.740375 12651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:40.740439 12651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:40.740517 12651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:40.744558 12651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:40.744632 12651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:40.744670 12651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:40.744705 12651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:40.744738 12651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:40.745630 12642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } attempt: 2
I20260810 22:43:40.746618 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.746183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57902)
0810 22:43:40.746411 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.746500 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 22:43:40.747478 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.747247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57906)
0810 22:43:40.747362 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.747413 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:43:40.747700 12642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:42461: Network error: Client connection negotiation failed: client connection to 127.19.202.125:42461: connect: Connection refused (error 111)
I20260810 22:43:40.750412 12651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:46645,127.19.202.125:42461,127.19.202.124:33223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab
--principal=kudu/127.19.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:42461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:40.751462 12651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:40.752461 12651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:40.760640 12657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:40.760650 12658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:40.760625 12661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:40.761341 12651 server_base.cc:1034] running on GCE node
I20260810 22:43:40.761953 12651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:40.763010 12651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:40.764189 12651 hybrid_clock.cc:648] HybridClock initialized: now 1786401820764168 us; error 35 us; skew 500 ppm
Aug 10 22:43:40 dist-test-slave-1x4g krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:40.767361 12651 init.cc:377] Logged in from keytab as kudu/127.19.202.125@KRBTEST.COM (short username kudu)
I20260810 22:43:40.768618 12651 webserver.cc:460] Webserver started at http://127.19.202.125:45015/ using document root <none> and password file <none>
I20260810 22:43:40.769361 12651 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:40.769462 12651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:40.769706 12651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:40.772640 12651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data/instance:
uuid: "1cb4ef8b5085435c8ff39449a79f550a"
format_stamp: "Formatted at 2026-08-10 22:43:40 on dist-test-slave-1x4g"
server_key: "547094bbf2ddc998113c3acc7d285b74"
server_key_iv: "9862bd450a5c66a32a53099954832c48"
server_key_version: "encryptionkey@0"
I20260810 22:43:40.773466 12651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/wal/instance:
uuid: "1cb4ef8b5085435c8ff39449a79f550a"
format_stamp: "Formatted at 2026-08-10 22:43:40 on dist-test-slave-1x4g"
server_key: "547094bbf2ddc998113c3acc7d285b74"
server_key_iv: "9862bd450a5c66a32a53099954832c48"
server_key_version: "encryptionkey@0"
I20260810 22:43:40.778668 12651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 22:43:40.782006 12669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:40.783630 12651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:40.783775 12651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/wal
uuid: "1cb4ef8b5085435c8ff39449a79f550a"
format_stamp: "Formatted at 2026-08-10 22:43:40 on dist-test-slave-1x4g"
server_key: "547094bbf2ddc998113c3acc7d285b74"
server_key_iv: "9862bd450a5c66a32a53099954832c48"
server_key_version: "encryptionkey@0"
I20260810 22:43:40.783910 12651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:40.804804 12651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:40.808640 12651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:40.808871 12651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:40.819250 12651 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.125:42461
I20260810 22:43:40.819531 12729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.125:42461 every 8 connection(s)
I20260810 22:43:40.820665 12651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
I20260810 22:43:40.823621 12730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:40.824165 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 12651
I20260810 22:43:40.824306 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-1/wal/instance
I20260810 22:43:40.824625 20265 external_mini_cluster.cc:1428] Setting key 7e5abe91d8f7e3b23b1610e65702715e
WARNING: no policy specified for kudu/127.19.202.124@KRBTEST.COM; defaulting to no policy
I20260810 22:43:40.838874 12642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } attempt: 3
I20260810 22:43:40.839938 12730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.124@KRBTEST.COM" created.
Aug 10 22:43:40 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Aug 10 22:43:40 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:40.857224 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.839455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57914)
0810 22:43:40.840046 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.840064 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:40.840178 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:40.843858 (+ 3680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:40.843864 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:40.844110 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:40.844266 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.844274 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.846456 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.846461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.847302 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.847313 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.848118 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.848121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.848257 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.848863 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:40.848877 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:40.851113 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:40.854809 (+ 3696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.854815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.854819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.855116 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.855552 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.855557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.855560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.855660 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.856512 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:40.856518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:40.856686 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 22:43:40.856808 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 22:43:40.856984 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":22}
I20260810 22:43:40.857479 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.842679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.19.202.126:46645)
0810 22:43:40.842832 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:40.842838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:40.842857 (+ 19us) server_negotiation.cc:407] Connection header received
0810 22:43:40.844704 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:40.844710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:40.844793 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:40.844884 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:40.845736 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.846456 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.847845 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.848540 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.848639 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.852887 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:40.852894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:40.852911 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:40.852944 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:40.854801 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.855501 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.855506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.855509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.855602 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.856009 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.856014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.856017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.856288 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:40.856405 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:40.857071 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 22:43:40.857228 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":297,"server-negotiator.queue_time_us":59}
I20260810 22:43:40.858973 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.840561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.19.202.125:42461)
0810 22:43:40.841236 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:40.841261 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:40.841309 (+ 48us) server_negotiation.cc:407] Connection header received
0810 22:43:40.843394 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:40.843419 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:40.843673 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:40.844446 (+ 773us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:40.844893 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.846332 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.847448 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.847992 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.851252 (+ 3260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.851425 (+ 173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:40.851434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:40.851450 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:40.851474 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:40.854671 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.855269 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.855277 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.855281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.855428 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.855780 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.855783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.855784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.856367 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:40.857437 (+ 1070us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:40.857568 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 22:43:40.857877 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":178,"threads_started":1}
I20260810 22:43:40.858973 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.842212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:46645 (local address 127.0.0.1:60786)
0810 22:43:40.842596 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.842645 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:40.843413 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:40.844945 (+ 1532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:40.844950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:40.845373 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:40.845581 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.845595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.846588 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.846590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.847717 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.847725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.848680 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.848684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.848840 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.849383 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:40.849420 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:40.852263 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:40.854948 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.854960 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.854974 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.855372 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.855723 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.855728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.855733 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.855880 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.856498 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:40.856505 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:40.856758 (+ 253us) client_negotiation.cc:769] Sending connection context
0810 22:43:40.857890 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0810 22:43:40.858056 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"spinlock_wait_cycles":9600,"thread_start_us":122,"threads_started":1}
I20260810 22:43:40.860945 12730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:40.861383 12642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:40.862206 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.861862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45062)
0810 22:43:40.862060 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.862123 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 22:43:40.863168 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.862788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45074)
0810 22:43:40.863025 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.863086 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260810 22:43:40.863415 12642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:33223: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Aug 10 22:43:40 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab.
Entry for principal kudu/127.19.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab.
I20260810 22:43:40.867378 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:33223
--webserver_interface=127.19.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab
--principal=kudu/127.19.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:46645,127.19.202.125:42461,127.19.202.124:33223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:43:40.870163 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.861590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57918)
0810 22:43:40.861846 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.861861 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:40.861948 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:40.862169 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:40.862172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:40.862362 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:40.862516 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.862523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.863352 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.863358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.863951 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:40.863991 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.864424 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.864427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:40.864511 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.865029 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:40.865039 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:40.866360 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:40.868623 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.868629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.868634 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.868964 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.869263 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:40.869268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:40.869270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:40.869341 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:40.869698 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:40.869701 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:40.869775 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 22:43:40.869871 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 22:43:40.870001 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260810 22:43:40.870174 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.861833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57918 (local address 127.19.202.125:42461)
0810 22:43:40.861922 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:40.861926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:40.861938 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:43:40.862017 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:40.862021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:40.862073 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:40.862158 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:40.862638 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.863262 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.864084 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:40.864317 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:40.864419 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:40.866467 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:40.866470 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:40.866473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:40.866493 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:40.868499 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.869096 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.869100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.869103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.869173 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:40.869444 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:40.869447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:40.869448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:40.869604 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:40.869698 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:40.869921 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 22:43:40.870055 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":38}
I20260810 22:43:40.870862 12730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:40.871625 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.871289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45078)
0810 22:43:40.871460 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.871545 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260810 22:43:40.872279 12673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
I20260810 22:43:40.873392 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.873051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45088)
0810 22:43:40.873249 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.873302 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260810 22:43:40.873589 12730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:33223: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
I20260810 22:43:40.894586 12642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } attempt: 1
I20260810 22:43:40.895463 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.895123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45092)
0810 22:43:40.895286 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.895354 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 22:43:40.896296 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.896013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45096)
0810 22:43:40.896143 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.896209 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 22:43:40.896495 12642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:33223: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
I20260810 22:43:40.922890 12730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } attempt: 1
I20260810 22:43:40.923835 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.923482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45104)
0810 22:43:40.923655 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.923726 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 22:43:40.924604 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.924380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45108)
0810 22:43:40.924492 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.924544 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 22:43:40.924770 12730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:33223: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
I20260810 22:43:40.971654 12642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } attempt: 2
I20260810 22:43:40.972499 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.972196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45110)
0810 22:43:40.972364 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.972422 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 22:43:40.973227 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.973038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45118)
0810 22:43:40.973132 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.973178 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 22:43:40.973366 12642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:33223: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
W20260810 22:43:40.983350 12736 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 22:43:40.983608 12736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:40.983654 12736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:40.983731 12736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:40.987579 12736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:40.987622 12736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:40.987640 12736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:40.987676 12736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:40.987707 12736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:40.992719 12736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:46645,127.19.202.125:42461,127.19.202.124:33223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab
--principal=kudu/127.19.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:33223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:40.992997 12730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } attempt: 2
I20260810 22:43:40.993723 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.993457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45132)
0810 22:43:40.993602 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.993657 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 22:43:40.993973 12736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:40.994434 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:40.994231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45142)
0810 22:43:40.994334 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:40.994382 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 22:43:40.994585 12730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:33223: Network error: Client connection negotiation failed: client connection to 127.19.202.124:33223: connect: Connection refused (error 111)
I20260810 22:43:40.995018 12736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:41.003358 12744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:41.003372 12743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:41.003427 12746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:41.003546 12736 server_base.cc:1034] running on GCE node
I20260810 22:43:41.004212 12736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:41.005299 12736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:41.006455 12736 hybrid_clock.cc:648] HybridClock initialized: now 1786401821006425 us; error 40 us; skew 500 ppm
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401821, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:41.009238 12736 init.cc:377] Logged in from keytab as kudu/127.19.202.124@KRBTEST.COM (short username kudu)
I20260810 22:43:41.010097 12736 webserver.cc:460] Webserver started at http://127.19.202.124:38537/ using document root <none> and password file <none>
I20260810 22:43:41.010602 12736 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:41.010658 12736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:41.010816 12736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:41.012751 12736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data/instance:
uuid: "2b27b003e0484058a3a5d7a0a5e71bff"
format_stamp: "Formatted at 2026-08-10 22:43:41 on dist-test-slave-1x4g"
server_key: "d8419ab7c1f365ec6215c4de5ba5fe9c"
server_key_iv: "78d1178eec47ffb85f6ba52bc2dde5bc"
server_key_version: "encryptionkey@0"
I20260810 22:43:41.013320 12736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/wal/instance:
uuid: "2b27b003e0484058a3a5d7a0a5e71bff"
format_stamp: "Formatted at 2026-08-10 22:43:41 on dist-test-slave-1x4g"
server_key: "d8419ab7c1f365ec6215c4de5ba5fe9c"
server_key_iv: "78d1178eec47ffb85f6ba52bc2dde5bc"
server_key_version: "encryptionkey@0"
I20260810 22:43:41.016907 12736 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 22:43:41.019150 12754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:41.020299 12736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:41.020431 12736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/wal
uuid: "2b27b003e0484058a3a5d7a0a5e71bff"
format_stamp: "Formatted at 2026-08-10 22:43:41 on dist-test-slave-1x4g"
server_key: "d8419ab7c1f365ec6215c4de5ba5fe9c"
server_key_iv: "78d1178eec47ffb85f6ba52bc2dde5bc"
server_key_version: "encryptionkey@0"
I20260810 22:43:41.020531 12736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:41.030982 12736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:41.034050 12736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:41.034248 12736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:41.041327 12736 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.124:33223
I20260810 22:43:41.041615 12816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.124:33223 every 8 connection(s)
I20260810 22:43:41.042309 12736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
I20260810 22:43:41.045195 12817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:41.045567 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 12736
I20260810 22:43:41.045701 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786401536977653-20265-0/minicluster-data/master-2/wal/instance
I20260810 22:43:41.046020 20265 external_mini_cluster.cc:1428] Setting key f26bb09debd94fc6483feef4718fd4b6
I20260810 22:43:41.050160 12817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401821, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:41.060467 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.047310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.19.202.124:33223)
0810 22:43:41.047714 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.047729 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.047762 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:43:41.048438 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.048454 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.048753 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.049109 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.050032 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.051065 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.052113 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.052586 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.052752 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.055243 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.055248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.055256 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.055280 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.057319 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.057722 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.057730 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.057737 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.057839 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.058138 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.058143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.058145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.058622 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.058814 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.059150 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.059440 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":131,"threads_started":1}
I20260810 22:43:41.061086 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.051894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:46645 (local address 127.0.0.1:60800)
0810 22:43:41.052677 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:41.052713 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:41.052867 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:41.053152 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:41.053157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:41.053501 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:41.053626 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.053632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.054365 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.054381 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.055023 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.055032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.055454 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.055457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.055574 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.056010 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:41.056021 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:41.057618 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:41.059498 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.059505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.059511 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.059775 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.060017 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.060020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.060022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.060116 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.060456 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:41.060465 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:41.060585 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 22:43:41.060768 (+ 183us) client_negotiation.cc:240] Negotiation successful
0810 22:43:41.060954 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":146,"threads_started":1}
I20260810 22:43:41.061144 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.051986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.19.202.126:46645)
0810 22:43:41.052128 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.052132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.052809 (+ 677us) server_negotiation.cc:407] Connection header received
0810 22:43:41.052972 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.052975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.053019 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.053102 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.053760 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.054249 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.055136 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.055344 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.055450 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.057767 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.057770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.057772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.057794 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.059406 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.059863 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.059865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.059866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.059936 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.060207 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.060209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.060210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.060361 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.060438 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.060864 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.060997 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:43:41.062564 12817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401821, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:41.071319 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.063030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:42461 (local address 127.0.0.1:57924)
0810 22:43:41.063154 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:41.063169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:41.063259 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:41.063486 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:41.063488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:41.063667 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:41.063796 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.063804 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.064657 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.064660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.065378 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.065389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.065827 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.065831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.065927 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.066370 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:41.066381 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:41.067893 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:41.069846 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.069849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.069850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.070085 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.070356 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.070364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.070366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.070457 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.070861 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:41.070864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:41.070988 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 22:43:41.071090 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 22:43:41.071188 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 22:43:41.071427 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.063134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57924 (local address 127.19.202.125:42461)
0810 22:43:41.063274 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.063279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.063293 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:43:41.063338 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.063340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.063378 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.063468 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.063928 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.064552 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.065488 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.065722 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.065795 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.068024 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.068027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.068030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.068057 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.069745 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.070180 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.070185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.070188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.070267 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.070547 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.070549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.070550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.070759 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.070861 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.071157 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.071264 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 22:43:41.072224 12817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:41.072374 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.062868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.19.202.126:46645)
0810 22:43:41.062981 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.062985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.063064 (+ 79us) server_negotiation.cc:407] Connection header received
0810 22:43:41.063451 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.063457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.063497 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.063573 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.064242 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.064925 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.065855 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.066068 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.066168 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.068756 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.068761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.068765 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.068821 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.070255 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.070694 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.070697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.070699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.070773 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.071035 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.071037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.071038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.071246 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.071822 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.072114 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.072231 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 22:43:41.072517 12642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } attempt: 3
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401821, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
I20260810 22:43:41.083603 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.073138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45166)
0810 22:43:41.073267 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:41.073283 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:41.073360 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:41.073598 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:41.073602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:41.073772 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:41.073870 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.073876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.075317 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.075320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.076120 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.076129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.076809 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.076813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.076882 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.077346 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:41.077361 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:41.079427 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:41.081753 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.081757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.081759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.082043 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.082329 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.082334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.082451 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.083102 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:41.083104 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:41.083204 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 22:43:41.083301 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 22:43:41.083430 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":20,"spinlock_wait_cycles":768}
I20260810 22:43:41.083829 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.072960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45160)
0810 22:43:41.073067 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:41.073093 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:41.073184 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:41.073865 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:41.073868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:41.074044 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:41.074161 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.074167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.075502 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.075505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.076055 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.076062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.076544 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.076547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.076685 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.077177 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:41.077202 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:41.079010 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:41.082216 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.082221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.082470 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.082790 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.082795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.082889 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.083301 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:41.083306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:41.083398 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 22:43:41.083510 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 22:43:41.083646 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 22:43:41.083848 12822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.073251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45160 (local address 127.19.202.124:33223)
0810 22:43:41.073610 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.073630 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.073645 (+ 15us) server_negotiation.cc:407] Connection header received
0810 22:43:41.073673 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.073677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.073720 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.074601 (+ 881us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.074684 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.075402 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.076180 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.076439 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.076541 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.079750 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.079753 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.079754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.079775 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.082072 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082589 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.082593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.082597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.082670 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082982 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.082985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.082987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.083199 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.083296 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.083570 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.083678 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":299,"spinlock_wait_cycles":896,"thread_start_us":140,"threads_started":1}
I20260810 22:43:41.084120 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.073198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.19.202.124:33223)
0810 22:43:41.073288 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.073292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.073327 (+ 35us) server_negotiation.cc:407] Connection header received
0810 22:43:41.073449 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.073454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.073494 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.073569 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.073997 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.075229 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.076236 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.076612 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.076699 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.079571 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.079574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.079576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.079597 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.081633 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082135 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.082139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.082141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.082220 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082553 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.082558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.082563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.082802 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.082900 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.083863 (+ 963us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.083974 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 22:43:41.084885 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.073883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.19.202.125:42461)
0810 22:43:41.073990 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.073993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.074426 (+ 433us) server_negotiation.cc:407] Connection header received
0810 22:43:41.074556 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.074558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.074589 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.074671 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.075241 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.075980 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.077096 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.077391 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.077498 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.080297 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.080303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.080306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.080333 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.082248 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.082707 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.082712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.082714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.082794 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.083077 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.083079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.083080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.083238 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.083486 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.084583 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.084675 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 22:43:41.085733 12730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } attempt: 3
I20260810 22:43:41.087590 12642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160: Bootstrap starting.
I20260810 22:43:41.088052 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:41.090633 12642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:41.089469 12613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d6406f11439642f0a16068d96a6ad160"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60810
Aug 10 22:43:41 dist-test-slave-1x4g krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401820, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
I20260810 22:43:41.091643 12642 log.cc:824] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:41.091753 12817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: Bootstrap starting.
I20260810 22:43:41.093924 12642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160: No bootstrap required, opened a new log
I20260810 22:43:41.094419 12817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:41.095256 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.086081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:33223 (local address 127.0.0.1:45174)
0810 22:43:41.086419 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:41.086437 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:41.086529 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:41.086880 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:41.086883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:41.087057 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:41.087146 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.087152 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.088054 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.088058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.088807 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:41.088819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.089491 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.089495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:41.089577 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.090043 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:41.090057 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:41.091569 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:41.093580 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.093583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.093586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.093825 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.094134 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:41.094137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:41.094140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:41.094236 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:41.094715 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:41.094718 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:41.094828 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 22:43:41.094948 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 22:43:41.095082 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260810 22:43:41.095263 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.086111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45174 (local address 127.19.202.124:33223)
0810 22:43:41.086204 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.086208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.086490 (+ 282us) server_negotiation.cc:407] Connection header received
0810 22:43:41.086722 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.086725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.086769 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.086846 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.087247 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.087943 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.089011 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.089363 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.089782 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.091714 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.091740 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.091742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.091768 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.093477 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.093949 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.093953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.093955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.094040 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.094334 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.094337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.094339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.094561 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.094694 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.095021 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.095129 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 22:43:41.095259 12817 log.cc:824] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:41.097545 12817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: No bootstrap required, opened a new log
I20260810 22:43:41.097805 12642 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.098033 12642 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6406f11439642f0a16068d96a6ad160, State: Initialized, Role: FOLLOWER
I20260810 22:43:41.098344 12730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: Bootstrap starting.
I20260810 22:43:41.098357 12642 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.098922 12826 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.099216 12826 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:41.099709 12642 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:41.099769 12613 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 22:43:41.099882 12613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:41.100855 12730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:41.101147 12613 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.101716 12730 log.cc:824] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:41.101678 12817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.101904 12817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b27b003e0484058a3a5d7a0a5e71bff, State: Initialized, Role: FOLLOWER
I20260810 22:43:41.102061 12613 leader_election.cc:288] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [CANDIDATE]: Term 1 election: Requested vote from peers 1cb4ef8b5085435c8ff39449a79f550a (127.19.202.125:42461), 2b27b003e0484058a3a5d7a0a5e71bff (127.19.202.124:33223)
I20260810 22:43:41.102200 12817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.103492 12817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:41.103544 12786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6406f11439642f0a16068d96a6ad160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2b27b003e0484058a3a5d7a0a5e71bff"
I20260810 22:43:41.103745 12786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:41.104118 12730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: No bootstrap required, opened a new log
I20260810 22:43:41.105337 12786 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6406f11439642f0a16068d96a6ad160 in term 1.
I20260810 22:43:41.105805 12584 leader_election.cc:302] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b27b003e0484058a3a5d7a0a5e71bff, d6406f11439642f0a16068d96a6ad160; no voters:
I20260810 22:43:41.108193 12730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.108387 12730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb4ef8b5085435c8ff39449a79f550a, State: Initialized, Role: FOLLOWER
I20260810 22:43:41.108664 12730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.108951 12839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.109069 12839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:41.109927 12730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:41.107906 12564 master_runner.cc:186] Error getting master registration for 127.19.202.125:42461: OK, instance_id { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" instance_seqno: 1786401820816405 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 22:43:41.111449 12564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 22:43:41.111572 12700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6406f11439642f0a16068d96a6ad160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1cb4ef8b5085435c8ff39449a79f550a"
I20260810 22:43:41.111793 12700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [term 0 FOLLOWER]: Advancing to term 1
W20260810 22:43:41.111869 12857 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:41.111935 12857 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 22:43:41.112542 12837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:41.112602 12837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:41.113157 12700 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6406f11439642f0a16068d96a6ad160 in term 1.
I20260810 22:43:41.110172 12736 master_runner.cc:186] Error getting master registration for 127.19.202.125:42461: OK, instance_id { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" instance_seqno: 1786401820816405 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 22:43:41.114197 12736 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 22:43:41.114348 12826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:41.114601 12826 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [term 1 LEADER]: Becoming Leader. State: Replica: d6406f11439642f0a16068d96a6ad160, State: Running, Role: LEADER
I20260810 22:43:41.115022 12826 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } }
I20260810 22:43:41.117012 12826 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6406f11439642f0a16068d96a6ad160. Latest consensus state: current_term: 1 leader_uuid: "d6406f11439642f0a16068d96a6ad160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.117791 12826 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: This master's current role is: LEADER
W20260810 22:43:41.121843 12880 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:41.121932 12880 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:41.124351 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.113417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.19.202.124:33223)
0810 22:43:41.114060 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.114065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.114441 (+ 376us) server_negotiation.cc:407] Connection header received
0810 22:43:41.114646 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.114649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.114698 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.114779 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.115851 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.116674 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.117952 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.118325 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.118431 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.120410 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.120424 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.120427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.120453 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.122351 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.122850 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.122854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.122856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.122945 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.123258 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.123262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.123264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.123496 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.123600 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.124016 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.124177 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571}
I20260810 22:43:41.125018 12862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.125154 12862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:41.133775 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.114005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.19.202.125:42461)
0810 22:43:41.114149 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.114154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.122340 (+ 8186us) server_negotiation.cc:407] Connection header received
0810 22:43:41.124889 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.124892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.124956 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.125035 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.125833 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.126606 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.127825 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.128178 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.128251 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.130039 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.130044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.130060 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.130085 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.131799 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.132312 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.132316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.132318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.132404 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.132743 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.132747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.132748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.132999 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.133102 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.133505 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.133618 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:43:41.139878 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:41.113008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60812 (local address 127.19.202.126:46645)
0810 22:43:41.113118 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.113122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.121032 (+ 7910us) server_negotiation.cc:407] Connection header received
0810 22:43:41.121220 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.121223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.121270 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.121325 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.125769 (+ 4444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.126570 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.133733 (+ 7163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:41.134071 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:41.134150 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:41.136164 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:41.136168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:41.136171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:41.136199 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:41.137975 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.138472 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.138476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.138478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.138567 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:41.138873 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:41.138876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:41.138877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:41.139101 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:41.139208 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:41.139598 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 22:43:41.139718 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:43:41.141950 12882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:41.143407 12882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:41.150494 12786 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [term 1 FOLLOWER]: Refusing update from remote peer d6406f11439642f0a16068d96a6ad160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:41.151269 12878 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:41.155128 12839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6406f11439642f0a16068d96a6ad160. Latest consensus state: current_term: 1 leader_uuid: "d6406f11439642f0a16068d96a6ad160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.155237 12839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:41.158365 12700 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [term 1 FOLLOWER]: Refusing update from remote peer d6406f11439642f0a16068d96a6ad160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:41.158934 12878 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:41.162397 12862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6406f11439642f0a16068d96a6ad160. Latest consensus state: current_term: 1 leader_uuid: "d6406f11439642f0a16068d96a6ad160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.162503 12862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:41.168900 12826 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6406f11439642f0a16068d96a6ad160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.169023 12826 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:41.169305 12884 mvcc.cc:205] Tried to move back new op lower bound from 7317101859423903744 to 7317101859290947584. Current Snapshot: MvccSnapshot[applied={T|T < 7317101859423903744}]
I20260810 22:43:41.169744 12839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6406f11439642f0a16068d96a6ad160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.169862 12839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:41.172866 12878 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6406f11439642f0a16068d96a6ad160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.172992 12878 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160 [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:41.174162 12862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6406f11439642f0a16068d96a6ad160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6406f11439642f0a16068d96a6ad160" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 46645 } } peers { permanent_uuid: "1cb4ef8b5085435c8ff39449a79f550a" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 42461 } } peers { permanent_uuid: "2b27b003e0484058a3a5d7a0a5e71bff" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 33223 } } }
I20260810 22:43:41.174264 12862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:41.174954 12885 mvcc.cc:205] Tried to move back new op lower bound from 7317101859423903744 to 7317101859290947584. Current Snapshot: MvccSnapshot[applied={T|T < 7317101859423903744}]
I20260810 22:43:41.176996 12887 mvcc.cc:205] Tried to move back new op lower bound from 7317101859423903744 to 7317101859290947584. Current Snapshot: MvccSnapshot[applied={T|T < 7317101859423903744}]
I20260810 22:43:41.177497 12882 catalog_manager.cc:1347] Generated new cluster ID: 55fe35067d8e4a1bbdc17d6ed207e39e
I20260810 22:43:41.177559 12882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:41.222735 12882 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:41.224151 12882 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:41.244334 12882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6406f11439642f0a16068d96a6ad160: Generated new TSK 0
I20260810 22:43:41.245236 12882 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 22:43:41.285053 12646 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.283033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.19.202.126:46645)
0810 22:43:41.283163 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.283167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.283726 (+ 559us) server_negotiation.cc:407] Connection header received
0810 22:43:41.284025 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.284030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.284084 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.284144 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.284903 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60822: BlockingRecv error: recv got EOF from 127.0.0.1:60822 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 22:43:41.287225 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.286145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.19.202.125:42461)
0810 22:43:41.286268 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.286274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.286291 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:43:41.286454 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.286457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.286512 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.286564 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.287101 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57950: BlockingRecv error: recv got EOF from 127.0.0.1:57950 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 22:43:41.288482 12820 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.285765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45188 (local address 127.19.202.124:33223)
0810 22:43:41.286368 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.286372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.287597 (+ 1225us) server_negotiation.cc:407] Connection header received
0810 22:43:41.287751 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.287755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.287813 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.287920 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.288363 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45188: BlockingRecv error: recv got EOF from 127.0.0.1:45188 (error 108)
Metrics: {"server-negotiator.queue_time_us":499}
W20260810 22:43:41.292582 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.291055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.19.202.125:42461)
0810 22:43:41.291151 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.291156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.291626 (+ 470us) server_negotiation.cc:407] Connection header received
0810 22:43:41.291831 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.291834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.291919 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.291970 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.292485 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57966: BlockingRecv error: recv got EOF from 127.0.0.1:57966 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 22:43:41.297439 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.295845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57968 (local address 127.19.202.125:42461)
0810 22:43:41.295949 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.295953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.296473 (+ 520us) server_negotiation.cc:407] Connection header received
0810 22:43:41.296695 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.296698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.296753 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.296827 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.297344 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57968: BlockingRecv error: recv got EOF from 127.0.0.1:57968 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 22:43:41.303128 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.301600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.19.202.125:42461)
0810 22:43:41.301696 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.301700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.302182 (+ 482us) server_negotiation.cc:407] Connection header received
0810 22:43:41.302415 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.302418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.302469 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.302522 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.303036 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57978: BlockingRecv error: recv got EOF from 127.0.0.1:57978 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 22:43:41.310842 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.309232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.19.202.125:42461)
0810 22:43:41.309365 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.309370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.309957 (+ 587us) server_negotiation.cc:407] Connection header received
0810 22:43:41.310148 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.310151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.310200 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.310250 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.310752 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57994: BlockingRecv error: recv got EOF from 127.0.0.1:57994 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 22:43:41.322826 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.321195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.19.202.125:42461)
0810 22:43:41.321311 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.321315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.321867 (+ 552us) server_negotiation.cc:407] Connection header received
0810 22:43:41.322057 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.322060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.322109 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.322176 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.322714 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58006: BlockingRecv error: recv got EOF from 127.0.0.1:58006 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 22:43:41.342283 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.341201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.19.202.125:42461)
0810 22:43:41.341336 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.341341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.341502 (+ 161us) server_negotiation.cc:407] Connection header received
0810 22:43:41.341605 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.341608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.341653 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.341731 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.342170 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58020: BlockingRecv error: recv got EOF from 127.0.0.1:58020 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 22:43:41.377733 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.376293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.19.202.125:42461)
0810 22:43:41.376447 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.376452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.376647 (+ 195us) server_negotiation.cc:407] Connection header received
0810 22:43:41.376843 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.376846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.376903 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.377159 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.377605 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58026: BlockingRecv error: recv got EOF from 127.0.0.1:58026 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 22:43:41.445159 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.443944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.19.202.125:42461)
0810 22:43:41.444091 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.444095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.444289 (+ 194us) server_negotiation.cc:407] Connection header received
0810 22:43:41.444419 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.444421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.444472 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.444552 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.445030 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58034: BlockingRecv error: recv got EOF from 127.0.0.1:58034 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 22:43:41.576690 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.575387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.19.202.125:42461)
0810 22:43:41.575553 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.575558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.575693 (+ 135us) server_negotiation.cc:407] Connection header received
0810 22:43:41.575868 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.575871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.575937 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.576025 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.576509 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58036: BlockingRecv error: recv got EOF from 127.0.0.1:58036 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 22:43:41.836855 12733 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:41.835331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.19.202.125:42461)
0810 22:43:41.835469 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:41.835476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:41.835571 (+ 95us) server_negotiation.cc:407] Connection header received
0810 22:43:41.835809 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:41.835816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:41.835890 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:41.835951 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:41.836719 (+ 768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58052: BlockingRecv error: recv got EOF from 127.0.0.1:58052 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:43:42.113361 12857 catalog_manager.cc:1259] Loaded cluster ID: 55fe35067d8e4a1bbdc17d6ed207e39e
I20260810 22:43:42.113473 12857 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: loading cluster ID for follower catalog manager: success
I20260810 22:43:42.115176 12857 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: acquiring CA information for follower catalog manager: success
I20260810 22:43:42.116397 12857 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b27b003e0484058a3a5d7a0a5e71bff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 22:43:42.123107 12880 catalog_manager.cc:1259] Loaded cluster ID: 55fe35067d8e4a1bbdc17d6ed207e39e
I20260810 22:43:42.123164 12880 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: loading cluster ID for follower catalog manager: success
I20260810 22:43:42.124936 12880 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: acquiring CA information for follower catalog manager: success
I20260810 22:43:42.125978 12880 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cb4ef8b5085435c8ff39449a79f550a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 22:43:42.355218 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:42.350987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.19.202.125:42461)
0810 22:43:42.351260 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:42.351264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:42.351284 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:43:42.351428 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:42.351431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:42.351626 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:42.351757 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:42.352049 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.352760 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.353682 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.353912 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.354020 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:42.354339 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:42.354807 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:42.354943 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:42.355002 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 22:43:42.355058 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260810 22:43:42.362016 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:42.358310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.19.202.124:33223)
0810 22:43:42.358568 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:42.358572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:42.358586 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:43:42.358724 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:42.358726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:42.358844 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:42.358983 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:42.359174 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.359705 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.360502 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.360717 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.360840 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:42.361099 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:42.361585 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:42.361682 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:42.361739 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 22:43:42.361791 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
I20260810 22:43:42.368741 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:42.365490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.19.202.124:33223)
0810 22:43:42.365629 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:42.365633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:42.365707 (+ 74us) server_negotiation.cc:407] Connection header received
0810 22:43:42.365848 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:42.365851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:42.365914 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:42.365989 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:42.366257 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.366803 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.367639 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.367854 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.367960 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:42.368271 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:42.368360 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:42.368420 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:42.368557 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 22:43:42.368604 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 22:43:42.369123 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:42.365180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.19.202.125:42461)
0810 22:43:42.365313 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:42.365320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:42.365749 (+ 429us) server_negotiation.cc:407] Connection header received
0810 22:43:42.365899 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:42.365902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:42.365980 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:42.366050 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:42.366411 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.366888 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.368033 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.368279 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.368387 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:42.368645 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:42.368720 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:42.368763 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:42.368934 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 22:43:42.368989 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:43:42.370565 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:42.365101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.19.202.126:46645)
0810 22:43:42.365474 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:42.365479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:42.365713 (+ 234us) server_negotiation.cc:407] Connection header received
0810 22:43:42.365834 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:42.365837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:42.366011 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:42.366127 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:42.367042 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.367830 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.368879 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:42.369234 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:42.369328 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:42.369720 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:42.370192 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:42.370290 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:42.370354 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 22:43:42.370424 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":61,"threads_started":1}
I20260810 22:43:42.373492 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 12564
I20260810 22:43:42.384150 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 12651
I20260810 22:43:42.393347 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 12736
2026-08-10T22:43:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3194 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:43:42 dist-test-slave-1x4g krb5kdc[12992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:43:42 dist-test-slave-1x4g krb5kdc[12992](info): set up 2 sockets
Aug 10 22:43:42 dist-test-slave-1x4g krb5kdc[12992](info): commencing operation
krb5kdc: starting...
W20260810 22:43:43.493251 20265 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/test-user.keytab.
Aug 10 22:43:43 dist-test-slave-1x4g krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:43:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:43:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.202.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
Entry for principal kudu/127.19.202.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab.
I20260810 22:43:43.623142 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:33485
--webserver_interface=127.19.202.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:36651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:33485,127.19.202.125:40921,127.19.202.124:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:43:43.735524 13007 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 22:43:43.735805 13007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:43.735828 13007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:43.735924 13007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:43:43.739979 13007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:43.740031 13007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:43.740051 13007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:43.740067 13007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:43.740116 13007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:43.745200 13007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:36651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:33485,127.19.202.125:40921,127.19.202.124:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.126.keytab
--principal=kudu/127.19.202.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.126:33485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.202.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:43.746234 13007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:43.747229 13007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:43.755813 13015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:43.755828 13012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:43.755833 13013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:43.756084 13007 server_base.cc:1034] running on GCE node
I20260810 22:43:43.756704 13007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:43.757791 13007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:43.758970 13007 hybrid_clock.cc:648] HybridClock initialized: now 1786401823758944 us; error 48 us; skew 500 ppm
Aug 10 22:43:43 dist-test-slave-1x4g krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:43.761721 13007 init.cc:377] Logged in from keytab as kudu/127.19.202.126@KRBTEST.COM (short username kudu)
I20260810 22:43:43.762626 13007 webserver.cc:460] Webserver started at http://127.19.202.126:40197/ using document root <none> and password file <none>
I20260810 22:43:43.763180 13007 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:43.763285 13007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:43.763514 13007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:43.765601 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data/instance:
uuid: "c40539881a5a46d6971f45c4bd8eacde"
format_stamp: "Formatted at 2026-08-10 22:43:43 on dist-test-slave-1x4g"
server_key: "84a05e3f365000c368588406d4d32663"
server_key_iv: "7ffc20b51af2996beb42bf73f4b074eb"
server_key_version: "encryptionkey@0"
I20260810 22:43:43.766173 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/wal/instance:
uuid: "c40539881a5a46d6971f45c4bd8eacde"
format_stamp: "Formatted at 2026-08-10 22:43:43 on dist-test-slave-1x4g"
server_key: "84a05e3f365000c368588406d4d32663"
server_key_iv: "7ffc20b51af2996beb42bf73f4b074eb"
server_key_version: "encryptionkey@0"
I20260810 22:43:43.769958 13007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 22:43:43.772255 13023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:43.773481 13007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 22:43:43.773622 13007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/wal
uuid: "c40539881a5a46d6971f45c4bd8eacde"
format_stamp: "Formatted at 2026-08-10 22:43:43 on dist-test-slave-1x4g"
server_key: "84a05e3f365000c368588406d4d32663"
server_key_iv: "7ffc20b51af2996beb42bf73f4b074eb"
server_key_version: "encryptionkey@0"
I20260810 22:43:43.773729 13007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:43.793140 13007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:43.795982 13007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:43.796181 13007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:43.803596 13007 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.126:33485
I20260810 22:43:43.803895 13080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.126:33485 every 8 connection(s)
I20260810 22:43:43.804610 13007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/data/info.pb
I20260810 22:43:43.807416 13081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:43.809659 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 13007
I20260810 22:43:43.809759 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-0/wal/instance
I20260810 22:43:43.810074 20265 external_mini_cluster.cc:1428] Setting key ae8a74151c7a2ae94272ae2cfef90c49
I20260810 22:43:43.811348 13081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.202.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.125@KRBTEST.COM" created.
Aug 10 22:43:43 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:43.827461 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.813582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33485 (local address 127.0.0.1:45364)
0810 22:43:43.814038 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:43.814090 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:43.814909 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:43.815555 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:43.815560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:43.816004 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:43.816232 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:43.816244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:43.817518 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:43.817521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:43.818224 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:43.818232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:43.818919 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:43.818922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:43.819014 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:43.819586 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:43.819603 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:43.822484 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:43.824490 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:43.824499 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:43.824509 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:43.824732 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:43.825094 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:43.825097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:43.825098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:43.825208 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:43.825764 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:43.825771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:43.825909 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 22:43:43.826066 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 22:43:43.826495 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":157,"thread_start_us":127,"threads_started":1}
I20260810 22:43:43.827461 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.813583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.19.202.126:33485)
0810 22:43:43.814143 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:43.814157 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:43.814203 (+ 46us) server_negotiation.cc:407] Connection header received
0810 22:43:43.815198 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:43.815202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:43.815381 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:43.815558 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:43.816374 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:43.817407 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:43.818338 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:43.818773 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:43.818939 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:43.822635 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:43.822640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:43.822650 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:43.822670 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:43.824344 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:43.824885 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:43.824894 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:43.824895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:43.824996 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:43.825294 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:43.825299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:43.825301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:43.825652 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:43.825793 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:43.826139 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 22:43:43.826445 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":431,"spinlock_wait_cycles":9216,"thread_start_us":145,"threads_started":1}
I20260810 22:43:43.830150 13081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:43.830865 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.830610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42818)
0810 22:43:43.830725 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:43.830808 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 22:43:43.831053 13026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
I20260810 22:43:43.831833 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.831621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42824)
0810 22:43:43.831718 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:43.831783 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 22:43:43.831988 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:40921: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab.
Entry for principal kudu/127.19.202.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab.
I20260810 22:43:43.838068 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:40921
--webserver_interface=127.19.202.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab
--principal=kudu/127.19.202.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:36651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:33485,127.19.202.125:40921,127.19.202.124:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:43:43.881215 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } attempt: 1
I20260810 22:43:43.882148 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.881781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42830)
0810 22:43:43.881974 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:43.882052 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 22:43:43.882896 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.882663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42844)
0810 22:43:43.882776 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:43.882831 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 22:43:43.883059 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:40921: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
W20260810 22:43:43.949617 13088 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 22:43:43.949883 13088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:43.949906 13088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:43.949990 13088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 22:43:43.951260 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } attempt: 2
I20260810 22:43:43.952163 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.951828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42858)
0810 22:43:43.952000 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:43.952072 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 22:43:43.952916 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:43.952671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42862)
0810 22:43:43.952804 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:43.952855 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:43:43.953069 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.125:40921: Network error: Client connection negotiation failed: client connection to 127.19.202.125:40921: connect: Connection refused (error 111)
W20260810 22:43:43.954452 13088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:43.954496 13088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:43.954515 13088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:43.954559 13088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:43.954591 13088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:43.959832 13088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:36651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:33485,127.19.202.125:40921,127.19.202.124:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.125.keytab
--principal=kudu/127.19.202.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.125:40921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.202.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:43.960855 13088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:43.961823 13088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:43:43.970139 13094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:43.970017 13093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:43.970110 13097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:43.970470 13088 server_base.cc:1034] running on GCE node
I20260810 22:43:43.971009 13088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:43.972067 13088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:43.973250 13088 hybrid_clock.cc:648] HybridClock initialized: now 1786401823973234 us; error 30 us; skew 500 ppm
Aug 10 22:43:43 dist-test-slave-1x4g krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:43.976159 13088 init.cc:377] Logged in from keytab as kudu/127.19.202.125@KRBTEST.COM (short username kudu)
I20260810 22:43:43.977165 13088 webserver.cc:460] Webserver started at http://127.19.202.125:42407/ using document root <none> and password file <none>
I20260810 22:43:43.977779 13088 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:43.977857 13088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:43.978056 13088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:43.980105 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data/instance:
uuid: "c3dbfd537a074718a9190dccbbbc9e8d"
format_stamp: "Formatted at 2026-08-10 22:43:43 on dist-test-slave-1x4g"
server_key: "29fa700aebbb796c1c0c02e18c0b02cf"
server_key_iv: "d1ed53d8705056cbae67e556179dd748"
server_key_version: "encryptionkey@0"
I20260810 22:43:43.980602 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/wal/instance:
uuid: "c3dbfd537a074718a9190dccbbbc9e8d"
format_stamp: "Formatted at 2026-08-10 22:43:43 on dist-test-slave-1x4g"
server_key: "29fa700aebbb796c1c0c02e18c0b02cf"
server_key_iv: "d1ed53d8705056cbae67e556179dd748"
server_key_version: "encryptionkey@0"
I20260810 22:43:43.984163 13088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 22:43:43.986469 13105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:43.987679 13088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 22:43:43.987787 13088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/wal
uuid: "c3dbfd537a074718a9190dccbbbc9e8d"
format_stamp: "Formatted at 2026-08-10 22:43:43 on dist-test-slave-1x4g"
server_key: "29fa700aebbb796c1c0c02e18c0b02cf"
server_key_iv: "d1ed53d8705056cbae67e556179dd748"
server_key_version: "encryptionkey@0"
I20260810 22:43:43.987891 13088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:44.006829 13088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:44.010186 13088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:44.010342 13088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:44.017416 13088 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.125:40921
I20260810 22:43:44.017691 13165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.125:40921 every 8 connection(s)
I20260810 22:43:44.018429 13088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/data/info.pb
I20260810 22:43:44.021306 13166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:44.024360 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 13088
I20260810 22:43:44.024488 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-1/wal/instance
I20260810 22:43:44.024750 20265 external_mini_cluster.cc:1428] Setting key 03d05a20c1915346362628cba62128e5
I20260810 22:43:44.025071 13166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.202.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.202.124@KRBTEST.COM" created.
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:44.042234 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.026963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.19.202.126:33485)
0810 22:43:44.027126 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.027133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.027945 (+ 812us) server_negotiation.cc:407] Connection header received
0810 22:43:44.028942 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.028948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.029013 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.029117 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.030199 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.031102 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.032236 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.032624 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.032742 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.037501 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.037503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.037506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.037530 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.039830 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.040498 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.040500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.040501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.040589 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.041051 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.041054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.041055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.041228 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.041306 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.042023 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.042117 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:43:44.043469 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.027326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33485 (local address 127.0.0.1:45366)
0810 22:43:44.027774 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.027826 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.028665 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.029257 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.029263 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.029602 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.030049 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.030062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.031232 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.031235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.032109 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.032119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.032829 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.032835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.032996 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.033564 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.033581 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.037318 (+ 3737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.039969 (+ 2651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.039977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.039994 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.040387 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.040681 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.040687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.040691 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.040948 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.041342 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.041352 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.041702 (+ 350us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.041979 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.042349 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":122,"threads_started":1}
I20260810 22:43:44.044207 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } attempt: 3
I20260810 22:43:44.045457 13166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.202.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab.
Entry for principal kudu/127.19.202.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab.
I20260810 22:43:44.054217 20265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
/tmp/dist-test-taskx9HXXp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:43305
--webserver_interface=127.19.202.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab
--principal=kudu/127.19.202.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.202.84:36651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.202.126:33485,127.19.202.125:40921,127.19.202.124:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:43:44.055192 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.044589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42872)
0810 22:43:44.044734 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.044749 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.044865 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.045496 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.045500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.045674 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.045812 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.045819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.047095 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.047098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.047783 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.047792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.048465 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.048469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.048600 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.049137 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.049157 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.050622 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.053488 (+ 2866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.053491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.053493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.053755 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.054124 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.054127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.054128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.054221 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.054657 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.054660 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.054758 (+ 98us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.054864 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.055004 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 22:43:44.055379 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.044694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.19.202.125:40921)
0810 22:43:44.045118 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.045127 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.045144 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:43:44.045177 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.045181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.045346 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.045476 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.045934 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.046936 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.047895 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.048255 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.048469 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.050746 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.050756 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.050772 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.050803 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.053380 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.053882 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.053890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.053893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.054031 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.054296 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.054302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.054304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.054554 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.054689 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.054959 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.055200 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":346,"spinlock_wait_cycles":8576,"thread_start_us":165,"threads_started":1}
I20260810 22:43:44.055397 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.045909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42876)
0810 22:43:44.046026 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.046039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.046109 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.046605 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.046607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.046734 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.046822 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.046827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.047625 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.047642 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.048221 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.048230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.048682 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.048685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.048827 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.049289 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.049304 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.051122 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.053553 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.053556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.053558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.053776 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.054063 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.054066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.054068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.054152 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.054744 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.054749 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.054846 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.054944 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.055122 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":5}
I20260810 22:43:44.056021 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.046070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.19.202.125:40921)
0810 22:43:44.046401 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.046409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.046422 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:43:44.046454 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.046457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.046493 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.046584 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.046929 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.047497 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.048336 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.048559 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.048681 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.051273 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.051277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.051281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.051310 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.053449 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.053882 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.053890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.053891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.053973 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.054256 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.054259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.054262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.054603 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.054706 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.055676 (+ 970us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.055807 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":264,"thread_start_us":125,"threads_started":1}
I20260810 22:43:44.057948 13081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:44.058314 13166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:44.058665 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.058445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59590)
0810 22:43:44.058535 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.058587 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 22:43:44.059022 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.058732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59600)
0810 22:43:44.058868 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.058944 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260810 22:43:44.059198 13106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
I20260810 22:43:44.059487 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.059154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59608)
0810 22:43:44.059337 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.059401 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260810 22:43:44.059659 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43305: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
I20260810 22:43:44.060043 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.059784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59614)
0810 22:43:44.059904 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.059966 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 22:43:44.060199 13166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43305: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
I20260810 22:43:44.090837 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } attempt: 1
I20260810 22:43:44.091749 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.091409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59624)
0810 22:43:44.091581 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.091665 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 22:43:44.092605 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.092329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59630)
0810 22:43:44.092462 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.092530 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 22:43:44.092830 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43305: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
I20260810 22:43:44.109460 13166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } attempt: 1
I20260810 22:43:44.110204 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.109907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59644)
0810 22:43:44.110065 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.110126 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 22:43:44.110996 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.110778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59656)
0810 22:43:44.110883 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.110934 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 22:43:44.111164 13166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43305: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
W20260810 22:43:44.166911 13174 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 22:43:44.167229 13174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:43:44.167297 13174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:43:44.167376 13174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 22:43:44.168058 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } attempt: 2
I20260810 22:43:44.168957 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.168611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59672)
0810 22:43:44.168753 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.168863 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 22:43:44.169667 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.169454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59684)
0810 22:43:44.169563 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.169616 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 22:43:44.169791 13081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43305: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
W20260810 22:43:44.171883 13174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:43:44.171963 13174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:43:44.172008 13174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:43:44.172044 13174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:43:44.172079 13174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:43:44.177222 13174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.202.84:36651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.202.126:33485,127.19.202.125:40921,127.19.202.124:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/krb5kdc/kudu_127.19.202.124.keytab
--principal=kudu/127.19.202.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.202.124:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.202.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 21:55:49 UTC on ccycloud.quasar-ktihrd.root.comops.site
build id 4579236
I20260810 22:43:44.178303 13174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:43:44.179314 13174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:43:44.179315 13166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } attempt: 2
I20260810 22:43:44.180094 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.179791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59686)
0810 22:43:44.179950 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.180027 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 22:43:44.180819 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.180564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59696)
0810 22:43:44.180672 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.180721 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 22:43:44.180955 13166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.202.124:43305: Network error: Client connection negotiation failed: client connection to 127.19.202.124:43305: connect: Connection refused (error 111)
W20260810 22:43:44.187939 13182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:44.187976 13181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:43:44.187944 13185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:43:44.188321 13174 server_base.cc:1034] running on GCE node
I20260810 22:43:44.188869 13174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:43:44.189879 13174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:43:44.191057 13174 hybrid_clock.cc:648] HybridClock initialized: now 1786401824191036 us; error 30 us; skew 500 ppm
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401824, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:43:44.193753 13174 init.cc:377] Logged in from keytab as kudu/127.19.202.124@KRBTEST.COM (short username kudu)
I20260810 22:43:44.194602 13174 webserver.cc:460] Webserver started at http://127.19.202.124:33899/ using document root <none> and password file <none>
I20260810 22:43:44.195112 13174 fs_manager.cc:359] Metadata directory not provided
I20260810 22:43:44.195169 13174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:43:44.195329 13174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:43:44.197372 13174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data/instance:
uuid: "9365713698b247bf8b71848fe9cb6ca6"
format_stamp: "Formatted at 2026-08-10 22:43:44 on dist-test-slave-1x4g"
server_key: "c78ecf4c06cf4867b81163f646995054"
server_key_iv: "e6eb5ee0c00277e501fd14fda7db8e97"
server_key_version: "encryptionkey@0"
I20260810 22:43:44.197892 13174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/wal/instance:
uuid: "9365713698b247bf8b71848fe9cb6ca6"
format_stamp: "Formatted at 2026-08-10 22:43:44 on dist-test-slave-1x4g"
server_key: "c78ecf4c06cf4867b81163f646995054"
server_key_iv: "e6eb5ee0c00277e501fd14fda7db8e97"
server_key_version: "encryptionkey@0"
I20260810 22:43:44.201483 13174 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 22:43:44.203717 13194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:43:44.204923 13174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 22:43:44.205049 13174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data,/tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/wal
uuid: "9365713698b247bf8b71848fe9cb6ca6"
format_stamp: "Formatted at 2026-08-10 22:43:44 on dist-test-slave-1x4g"
server_key: "c78ecf4c06cf4867b81163f646995054"
server_key_iv: "e6eb5ee0c00277e501fd14fda7db8e97"
server_key_version: "encryptionkey@0"
I20260810 22:43:44.205150 13174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:43:44.241881 13174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:43:44.244868 13174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:43:44.245091 13174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:43:44.252408 13174 rpc_server.cc:307] RPC server started. Bound to: 127.19.202.124:43305
I20260810 22:43:44.252734 13254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.202.124:43305 every 8 connection(s)
I20260810 22:43:44.253487 13174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/data/info.pb
I20260810 22:43:44.256400 13255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:43:44.260450 13255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:44.261345 20265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu as pid 13174
I20260810 22:43:44.261463 20265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskx9HXXp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786401536977653-20265-0/minicluster-data/master-2/wal/instance
I20260810 22:43:44.261770 20265 external_mini_cluster.cc:1428] Setting key eda4e5662ce5624d923b49dc6cb37a7e
I20260810 22:43:44.268946 13081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } attempt: 3
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401824, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
I20260810 22:43:44.272109 13166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } attempt: 3
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.126@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
I20260810 22:43:44.275279 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.262630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45374 (local address 127.19.202.126:33485)
0810 22:43:44.262784 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.262788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.263677 (+ 889us) server_negotiation.cc:407] Connection header received
0810 22:43:44.264846 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.264849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.264899 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.264972 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.265919 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.266427 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.267499 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.267811 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.267891 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.270694 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.270697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.270702 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.270733 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.272500 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.273287 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.273292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.273294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.273393 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.273828 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.273831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.273847 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.274086 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.274156 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.275000 (+ 844us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.275115 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 22:43:44.277343 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.263062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.126:33485 (local address 127.0.0.1:45374)
0810 22:43:44.263536 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.263583 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.264571 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.265048 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.265054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.265498 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.265747 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.265783 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.266532 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.266535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.267385 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.267394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.267955 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.267959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.268135 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.268807 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.268836 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.270550 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.272618 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.272631 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.272644 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.273153 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.273522 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.273525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.273527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.273714 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.274272 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.274282 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.274596 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.275790 (+ 1194us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.276155 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":116,"threads_started":1}
I20260810 22:43:44.277357 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.263828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.19.202.124:43305)
0810 22:43:44.264081 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.264092 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.264131 (+ 39us) server_negotiation.cc:407] Connection header received
0810 22:43:44.264571 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.264594 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.264880 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.265073 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.265832 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.266850 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.267707 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.268133 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.268231 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.271330 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.271339 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.271354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.271385 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.273773 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.274337 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.274345 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.274347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.274455 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.274775 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.274778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.274780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.275089 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.275313 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.276072 (+ 759us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.276275 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.125@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
I20260810 22:43:44.279603 13255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:44.280491 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.269386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59702)
0810 22:43:44.269595 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.269611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.269691 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.270093 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.270099 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.270334 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.270460 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.270468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.271425 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.271432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.272255 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.272268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.272964 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.272969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.273064 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.273502 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.273520 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.274957 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.277574 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.277577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.277579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.277847 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.278187 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.278190 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.278192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.278279 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.279950 (+ 1671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.279953 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.280056 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.280172 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.280301 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260810 22:43:44.280593 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.269478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.19.202.124:43305)
0810 22:43:44.269819 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.269828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.269848 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:43:44.269881 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.269884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.269927 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.270055 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.270589 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.271306 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.272403 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.272768 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.273267 (+ 499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.275631 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.275640 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.275645 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.275669 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.277450 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.277982 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.277987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.277989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.278075 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.278395 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.278398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.278399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.278611 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.279960 (+ 1349us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.280248 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.280378 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":103,"threads_started":1}
I20260810 22:43:44.283131 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.273134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59716)
0810 22:43:44.273538 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.273552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.273628 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.274121 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.274126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.274355 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.274455 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.274462 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.275549 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.275554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.276915 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.276927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.277541 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.277548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.277648 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.278103 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.278122 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.279417 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.281365 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.281369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.281371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.281567 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.281885 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.281889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.281890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.281992 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.282640 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.282643 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.282747 (+ 104us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.282853 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.282978 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318}
I20260810 22:43:44.283591 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.272866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.19.202.124:43305)
0810 22:43:44.273881 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.273892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.273905 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:43:44.273936 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.273940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.273979 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.274567 (+ 588us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.274649 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.275393 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.277064 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.277422 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.277540 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.279613 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.279616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.279618 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.279641 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.281263 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.281666 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.281669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.281670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.281752 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.282095 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.282097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.282098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.282240 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.282339 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.283333 (+ 994us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.283452 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":103,"threads_started":1}
I20260810 22:43:44.285281 13081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde: Bootstrap starting.
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401824, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
I20260810 22:43:44.288164 13081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde: Neither blocks nor log segments found. Creating new log.
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.126@KRBTEST.COM
I20260810 22:43:44.288627 13166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: Bootstrap starting.
I20260810 22:43:44.289146 13081 log.cc:824] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:44.289777 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.280990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.19.202.125:40921)
0810 22:43:44.281098 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.281102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.281113 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:43:44.281148 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.281150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.281181 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.281276 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.281703 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.282451 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.283547 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.283965 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.284043 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.286204 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.286207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.286209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.286231 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.287894 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.288377 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.288380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.288382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.288462 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.288803 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.288807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.288808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.289060 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.289133 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.289513 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.289642 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:43:44.290236 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.280705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.125:40921 (local address 127.0.0.1:42880)
0810 22:43:44.280803 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.280816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.280900 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.281291 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.281294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.281474 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.281557 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.281564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.282588 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.282594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.283418 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.283430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.284200 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.284204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.284268 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.284697 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.284710 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.286058 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.288024 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.288026 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.288028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.288261 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.288576 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.288579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.288581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.288671 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.289212 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.289215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.289311 (+ 96us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.289935 (+ 624us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.290091 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 22:43:44.290951 13255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:43:44.291379 13081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde: No bootstrap required, opened a new log
I20260810 22:43:44.292546 13166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:44.292595 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.281797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45376 (local address 127.19.202.126:33485)
0810 22:43:44.281923 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.281926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.282482 (+ 556us) server_negotiation.cc:407] Connection header received
0810 22:43:44.282668 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.282671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.282710 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.282756 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.284149 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.284853 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.285832 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.286107 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.286176 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.289028 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.289031 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.289034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.289058 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.290678 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.291176 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.291179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.291181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.291266 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.291599 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.291602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.291603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.291808 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.291961 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.292311 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.292456 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 22:43:44.293783 13166 log.cc:824] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:43:44.295284 13081 raft_consensus.cc:348] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.295544 13081 raft_consensus.cc:729] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40539881a5a46d6971f45c4bd8eacde, State: Initialized, Role: FOLLOWER
I20260810 22:43:44.295872 13166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: No bootstrap required, opened a new log
I20260810 22:43:44.295885 13081 consensus_queue.cc:260] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.296444 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.296631 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: This master's current role is: FOLLOWER
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401824, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.202.124@KRBTEST.COM for kudu/127.19.202.124@KRBTEST.COM
I20260810 22:43:44.297652 13081 sys_catalog.cc:564] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: configured and running, proceeding with master startup.
W20260810 22:43:44.297848 13029 tablet.cc:2369] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:43:44.300272 13166 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.300510 13166 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3dbfd537a074718a9190dccbbbc9e8d, State: Initialized, Role: FOLLOWER
I20260810 22:43:44.300904 13166 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.302053 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.291690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.19.202.124:43305)
0810 22:43:44.291777 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.291814 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.292062 (+ 248us) server_negotiation.cc:407] Connection header received
0810 22:43:44.292086 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.292088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.292119 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.292174 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.292674 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.293441 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.294462 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.294808 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.294914 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.297500 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.297504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.297507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.297528 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.299778 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.300217 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.300220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.300222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.300298 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.301184 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.301191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.301195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.301424 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.301498 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.301796 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.301916 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 22:43:44.302276 13166 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:44.302726 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.291367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.202.124:43305 (local address 127.0.0.1:59732)
0810 22:43:44.291908 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0810 22:43:44.291921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:43:44.291995 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:43:44.292255 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:43:44.292258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:43:44.292446 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:43:44.292543 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.292550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.293557 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.293562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.294318 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:43:44.294328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.294935 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.294939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:43:44.295020 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.295493 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:43:44.295710 (+ 217us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:43:44.297352 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:43:44.299892 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.299895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.299897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.300122 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.300414 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:43:44.300416 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:43:44.300418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:43:44.301010 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:43:44.301557 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:43:44.301559 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:43:44.301636 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 22:43:44.302372 (+ 736us) client_negotiation.cc:240] Negotiation successful
0810 22:43:44.302560 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":98,"threads_started":1}
Aug 10 22:43:44 dist-test-slave-1x4g krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786401823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.202.125@KRBTEST.COM
I20260810 22:43:44.305532 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.305727 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:44.305750 13255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: Bootstrap starting.
I20260810 22:43:44.302706 13007 master_runner.cc:186] Error getting master registration for 127.19.202.125:40921: OK, instance_id { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" instance_seqno: 1786401824015414 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 22:43:44.307376 13007 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 22:43:44.308077 13255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: Neither blocks nor log segments found. Creating new log.
I20260810 22:43:44.308972 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.298094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.19.202.125:40921)
0810 22:43:44.298197 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.298201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.298702 (+ 501us) server_negotiation.cc:407] Connection header received
0810 22:43:44.298749 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.298752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.298792 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.298868 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.299814 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.300492 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.301567 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.301872 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.301963 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.304958 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.304962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.304965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.304989 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.306762 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.307232 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.307236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.307238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.307325 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.307648 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.307651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.307652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.307847 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.308041 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.308644 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.308759 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:43:44.308995 13255 log.cc:824] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: Log is configured to *not* fsync() on all Append() calls
W20260810 22:43:44.310530 13278 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:44.310607 13278 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:44.312031 20265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:43:44.313917 13052 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c40539881a5a46d6971f45c4bd8eacde"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45376
I20260810 22:43:44.314121 13052 raft_consensus.cc:482] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 22:43:44.314235 13052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [term 0 FOLLOWER]: Advancing to term 1
W20260810 22:43:44.314275 13294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:44.314352 13294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:44.315559 13052 raft_consensus.cc:504] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.317090 13133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c40539881a5a46d6971f45c4bd8eacde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c3dbfd537a074718a9190dccbbbc9e8d"
I20260810 22:43:44.317328 13133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:44.318693 13133 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40539881a5a46d6971f45c4bd8eacde in term 1.
I20260810 22:43:44.319111 13026 leader_election.cc:302] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3dbfd537a074718a9190dccbbbc9e8d, c40539881a5a46d6971f45c4bd8eacde; no voters:
I20260810 22:43:44.319419 13265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:43:44.319631 13265 raft_consensus.cc:686] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [term 1 LEADER]: Becoming Leader. State: Replica: c40539881a5a46d6971f45c4bd8eacde, State: Running, Role: LEADER
I20260810 22:43:44.320007 13265 consensus_queue.cc:237] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.324888 13052 leader_election.cc:288] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [CANDIDATE]: Term 1 election: Requested vote from peers c3dbfd537a074718a9190dccbbbc9e8d (127.19.202.125:40921), 9365713698b247bf8b71848fe9cb6ca6 (127.19.202.124:43305)
I20260810 22:43:44.322959 13088 master_runner.cc:186] Error getting master registration for 127.19.202.124:43305: OK, instance_id { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" instance_seqno: 1786401824250378 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 22:43:44.326756 13088 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 22:43:44.327591 13255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: No bootstrap required, opened a new log
I20260810 22:43:44.328898 13296 sys_catalog.cc:455] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: SysCatalogTable state changed. Reason: New leader c40539881a5a46d6971f45c4bd8eacde. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.329336 13296 sys_catalog.cc:458] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:44.335852 13255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.336328 13255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9365713698b247bf8b71848fe9cb6ca6, State: Initialized, Role: FOLLOWER
I20260810 22:43:44.337042 13305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:43:44.336936 13255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } }
I20260810 22:43:44.338054 13305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:43:44.339251 13226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c40539881a5a46d6971f45c4bd8eacde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9365713698b247bf8b71848fe9cb6ca6"
I20260810 22:43:44.339509 13226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:43:44.340912 13226 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40539881a5a46d6971f45c4bd8eacde in term 1.
I20260810 22:43:44.341554 13255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:43:44.347718 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.328663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.19.202.126:33485)
0810 22:43:44.328794 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.328799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.329933 (+ 1134us) server_negotiation.cc:407] Connection header received
0810 22:43:44.330099 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.330102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.330155 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.330199 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.331468 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.332269 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.337309 (+ 5040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.337648 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.337719 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.341715 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.341719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.341722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.341748 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.343466 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.343975 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.343981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.343984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.344069 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.344356 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.344360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.344362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.344680 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.344817 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.347432 (+ 2615us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.347557 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 22:43:44.348383 13319 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:43:44.348574 13319 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:43:44.352702 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.329355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.19.202.125:40921)
0810 22:43:44.329456 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.329460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.331306 (+ 1846us) server_negotiation.cc:407] Connection header received
0810 22:43:44.332411 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.332414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.332462 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.332510 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.335274 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.336072 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.339603 (+ 3531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.339958 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.340030 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.345143 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.345147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.345150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.345176 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.346866 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.350054 (+ 3188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.350058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.350060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.350151 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.350737 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.350740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.350742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.350968 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.351077 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.352411 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.352549 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:43:44.353466 13226 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [term 1 FOLLOWER]: Refusing update from remote peer c40539881a5a46d6971f45c4bd8eacde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:44.354103 13296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [LEADER]: Connected to new peer: Peer: permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:44.359182 13133 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [term 1 FOLLOWER]: Refusing update from remote peer c40539881a5a46d6971f45c4bd8eacde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:43:44.359754 13296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:43:44.361116 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:44.334867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.19.202.124:43305)
0810 22:43:44.334992 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.334997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.350393 (+ 15396us) server_negotiation.cc:407] Connection header received
0810 22:43:44.351391 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.351396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.351455 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.351533 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.353485 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.354171 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.355355 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:44.355689 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:44.355761 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:44.357509 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:43:44.357513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:43:44.357516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:43:44.357545 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:43:44.359249 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.359740 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.359746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.359748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.359848 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:43:44.360142 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:43:44.360145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:43:44.360147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:43:44.360356 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:43:44.360483 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:44.360854 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 22:43:44.360970 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 22:43:44.363781 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c40539881a5a46d6971f45c4bd8eacde. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.363910 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:44.365916 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.366034 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:44.366194 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.366313 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:44.366515 13308 mvcc.cc:205] Tried to move back new op lower bound from 7317101872544010240 to 7317101872418631680. Current Snapshot: MvccSnapshot[applied={T|T < 7317101872544010240}]
I20260810 22:43:44.366746 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.366883 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:44.366986 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c40539881a5a46d6971f45c4bd8eacde. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.367056 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:44.369961 13325 mvcc.cc:205] Tried to move back new op lower bound from 7317101872544010240 to 7317101872418631680. Current Snapshot: MvccSnapshot[applied={T|T < 7317101872544010240}]
I20260810 22:43:44.378304 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.378422 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde [sys.catalog]: This master's current role is: LEADER
I20260810 22:43:44.379859 13322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c40539881a5a46d6971f45c4bd8eacde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40539881a5a46d6971f45c4bd8eacde" member_type: VOTER last_known_addr { host: "127.19.202.126" port: 33485 } } peers { permanent_uuid: "c3dbfd537a074718a9190dccbbbc9e8d" member_type: VOTER last_known_addr { host: "127.19.202.125" port: 40921 } } peers { permanent_uuid: "9365713698b247bf8b71848fe9cb6ca6" member_type: VOTER last_known_addr { host: "127.19.202.124" port: 43305 } } }
I20260810 22:43:44.379989 13322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:43:44.380218 13305 catalog_manager.cc:1347] Generated new cluster ID: a8e94cbf100e4d96b40c766c832284e1
I20260810 22:43:44.380275 13305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:43:44.381201 13323 mvcc.cc:205] Tried to move back new op lower bound from 7317101872544010240 to 7317101872418631680. Current Snapshot: MvccSnapshot[applied={T|T < 7317101872544010240}]
I20260810 22:43:44.416277 13305 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:43:44.417708 13305 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:43:44.429186 13305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c40539881a5a46d6971f45c4bd8eacde: Generated new TSK 0
I20260810 22:43:44.430063 13305 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 22:43:44.467191 13261 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.463376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.19.202.124:43305)
0810 22:43:44.463512 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.463518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.465404 (+ 1886us) server_negotiation.cc:407] Connection header received
0810 22:43:44.466071 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.466075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.466128 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.466179 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.467051 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59748: BlockingRecv error: recv got EOF from 127.0.0.1:59748 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 22:43:44.467954 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.463862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.19.202.125:40921)
0810 22:43:44.463969 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.463973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.465606 (+ 1633us) server_negotiation.cc:407] Connection header received
0810 22:43:44.466314 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.466318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.466364 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.466410 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.467827 (+ 1417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42904: BlockingRecv error: recv got EOF from 127.0.0.1:42904 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 22:43:44.469019 13085 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.464252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.19.202.126:33485)
0810 22:43:44.464356 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.464360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.465988 (+ 1628us) server_negotiation.cc:407] Connection header received
0810 22:43:44.468286 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.468289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.468336 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.468385 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.468914 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45398: BlockingRecv error: recv got EOF from 127.0.0.1:45398 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 22:43:44.473304 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.471861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.19.202.125:40921)
0810 22:43:44.471949 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.471952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.472360 (+ 408us) server_negotiation.cc:407] Connection header received
0810 22:43:44.472588 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.472592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.472660 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.472709 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.473217 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42918: BlockingRecv error: recv got EOF from 127.0.0.1:42918 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260810 22:43:44.478391 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.476655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.19.202.125:40921)
0810 22:43:44.476750 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.476754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.477511 (+ 757us) server_negotiation.cc:407] Connection header received
0810 22:43:44.477688 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.477691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.477738 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.477786 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.478306 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42920: BlockingRecv error: recv got EOF from 127.0.0.1:42920 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 22:43:44.483788 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.482260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42928 (local address 127.19.202.125:40921)
0810 22:43:44.482355 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.482358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.482894 (+ 536us) server_negotiation.cc:407] Connection header received
0810 22:43:44.483085 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.483089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.483137 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.483187 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.483687 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42928: BlockingRecv error: recv got EOF from 127.0.0.1:42928 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 22:43:44.491680 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.489999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.19.202.125:40921)
0810 22:43:44.490117 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.490121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.490735 (+ 614us) server_negotiation.cc:407] Connection header received
0810 22:43:44.490909 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.490912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.490958 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.491006 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.491561 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42930: BlockingRecv error: recv got EOF from 127.0.0.1:42930 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 22:43:44.503386 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.501767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.19.202.125:40921)
0810 22:43:44.501872 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.501876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.502425 (+ 549us) server_negotiation.cc:407] Connection header received
0810 22:43:44.502652 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.502656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.502711 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.502764 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.503290 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42936: BlockingRecv error: recv got EOF from 127.0.0.1:42936 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 22:43:44.523175 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.521612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42944 (local address 127.19.202.125:40921)
0810 22:43:44.521715 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.521719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.522276 (+ 557us) server_negotiation.cc:407] Connection header received
0810 22:43:44.522471 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.522474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.522524 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.522573 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.523070 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42944: BlockingRecv error: recv got EOF from 127.0.0.1:42944 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 22:43:44.558600 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.557474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.19.202.125:40921)
0810 22:43:44.557616 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.557622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.557766 (+ 144us) server_negotiation.cc:407] Connection header received
0810 22:43:44.557908 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.557911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.557958 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.558036 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.558466 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42952: BlockingRecv error: recv got EOF from 127.0.0.1:42952 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 22:43:44.626003 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.624702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.19.202.125:40921)
0810 22:43:44.624892 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.624898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.625042 (+ 144us) server_negotiation.cc:407] Connection header received
0810 22:43:44.625213 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.625216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.625281 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.625393 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.625863 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42954: BlockingRecv error: recv got EOF from 127.0.0.1:42954 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 22:43:44.757591 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:44.756203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.19.202.125:40921)
0810 22:43:44.756367 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:44.756380 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:44.756510 (+ 130us) server_negotiation.cc:407] Connection header received
0810 22:43:44.756725 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:44.756729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:44.756837 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:44.756924 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:44.757456 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42962: BlockingRecv error: recv got EOF from 127.0.0.1:42962 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 22:43:45.017010 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:43:45.015658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.19.202.125:40921)
0810 22:43:45.015798 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:45.015803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:45.015976 (+ 173us) server_negotiation.cc:407] Connection header received
0810 22:43:45.016149 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:45.016153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:45.016203 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:45.016299 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:43:45.016885 (+ 586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42964: BlockingRecv error: recv got EOF from 127.0.0.1:42964 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:43:45.315685 13294 catalog_manager.cc:1259] Loaded cluster ID: a8e94cbf100e4d96b40c766c832284e1
I20260810 22:43:45.315780 13294 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: loading cluster ID for follower catalog manager: success
I20260810 22:43:45.317595 13294 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: acquiring CA information for follower catalog manager: success
I20260810 22:43:45.318779 13294 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c3dbfd537a074718a9190dccbbbc9e8d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 22:43:45.349948 13319 catalog_manager.cc:1259] Loaded cluster ID: a8e94cbf100e4d96b40c766c832284e1
I20260810 22:43:45.350035 13319 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: loading cluster ID for follower catalog manager: success
I20260810 22:43:45.351981 13319 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: acquiring CA information for follower catalog manager: success
I20260810 22:43:45.353251 13319 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9365713698b247bf8b71848fe9cb6ca6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 22:43:45.534806 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:45.531021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.19.202.125:40921)
0810 22:43:45.531269 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:45.531273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:45.531359 (+ 86us) server_negotiation.cc:407] Connection header received
0810 22:43:45.531516 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:45.531518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:45.531652 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:45.531752 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:45.531988 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.532501 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.533379 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.533605 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.533683 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:45.533953 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:45.534414 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:45.534521 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:45.534554 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 22:43:45.534602 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":66,"threads_started":1}
I20260810 22:43:45.541237 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:45.537683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.19.202.124:43305)
0810 22:43:45.537911 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:45.537927 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:45.537943 (+ 16us) server_negotiation.cc:407] Connection header received
0810 22:43:45.538036 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:45.538038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:45.538148 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:45.538265 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:45.538477 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.538992 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.539804 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.540030 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.540132 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:45.540419 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:45.540853 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:45.540987 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:45.541029 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 22:43:45.541080 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":61,"threads_started":1}
I20260810 22:43:45.548161 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:45.544009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.19.202.125:40921)
0810 22:43:45.544121 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:45.544125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:45.544849 (+ 724us) server_negotiation.cc:407] Connection header received
0810 22:43:45.544945 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:45.544948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:45.544997 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:45.545077 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:45.545349 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.545988 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.547055 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.547379 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.547481 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:45.547739 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:45.547812 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:45.547887 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:45.547993 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 22:43:45.548041 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:43:45.548636 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:45.544477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.19.202.124:43305)
0810 22:43:45.544574 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:45.544577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:45.545138 (+ 561us) server_negotiation.cc:407] Connection header received
0810 22:43:45.545311 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:45.545316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:45.545364 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:45.545411 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:45.545758 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.546443 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.547385 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.547687 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.547755 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:45.548080 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:45.548146 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:45.548187 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:45.548456 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 22:43:45.548514 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:43:45.548923 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:43:45.544148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.19.202.126:33485)
0810 22:43:45.544399 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 22:43:45.544403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:43:45.544814 (+ 411us) server_negotiation.cc:407] Connection header received
0810 22:43:45.544989 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:43:45.544992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:43:45.545141 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:43:45.545271 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:43:45.545518 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.546251 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.547308 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:43:45.547598 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:43:45.547704 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:43:45.548019 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:43:45.548319 (+ 300us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:43:45.548394 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:43:45.548530 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 22:43:45.548589 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":63,"threads_started":1}
I20260810 22:43:45.555964 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 13007
I20260810 22:43:45.565937 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 13088
I20260810 22:43:45.573957 20265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskx9HXXp/build/debug/bin/kudu with pid 13174
2026-08-10T22:43:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3178 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6373 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288565 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST